Getting Started
Welcome to iData 2
If you are new to data quality assurance and data management, then perhaps this is the best place for you start.
iData 2 offers a range of facilities to help assess, report on, manage and provide data quality, none production data, and transformation.
The product is a web service, that can connect to your data sources, and analyse them in situ, ingest them into temporary holding for experimentation, transition them to a different destination all at your configuration.
The processes are set out as what we refer to as Data Processing Stages, or just 'Stages'.
There are five basic types of stages, each design to perform a separate set of functionality.
Profiling | Profiling is the analysis engine that can sample your data, and report on key indicators on data quality or formatting. Typically identifying poor email addresses, blanks or nulls in important fields. | |
Validate and Clean | Validate and Clean, sometimes referred to a ETL, is used to transform data. It's typically used when taking in an initial data set (ingesting), reporting on known data conditions that are flagged from profiling, running data cleansing, and performing data masking and obfuscation. | |
Similar Records | Similar Records, or sometimes referred to as Duplicate Records, is used to pattern match similarities across records within the same data set or source to indicate if there a duplicates in the data. An example might be where a web site permits registrations and some customers register multiple times with slightly different spellings of their names. | |
Data Generation | Data Generation stages are used to create synthetic data that meets you data structure and schema. The process can create rows of data across multiple data entities (tables) and maintain data referential integrity, insofar as keeping key values maintained across the destination tables. | |
Comparison and Assurance | Comparison and Assurance offers the facilities to hold two data sources side by side and identify where they are different. This includes records included in either source or destination and missing in the other set as well as field content differences. The stage can also accept modelling difference rules to report on or ignore specific cases. For example post a data cleans double spaces may have been replaced with singles, these replacements can be excluded from the report. |
Introduction Guides
As part of our support knowledge base, we have written some introductions to the basic functionality of the product. We recommend that you take the time to view these, in sequence.
Related Articles
Introduction to Similar Records
Prerequisites We start this topic with a Project created, a Data Source and Data Entities have been defined. We will be using the AcmeData project we created in previous articles. Video Introduction In this short article we will be: Creating a ...What to Expect From an iData 2 Beta or Proof of Concept?
Introduction The wait is almost over! IDS are soft launching our iData 2.0 product. The all-new iData 2.0, is easier, faster, more intuitive and more collaborative to use: New workflow-driven user interface reduces the need for tech support ...Introduction and Projects
Prerequisites We start this topic with a Clean iData instance for our user. No projects are available. Video Introduction In this short article we will be: Introducing the initial screens of the iData product and creating a new iData project. Steps ...How Does iData Perform End-2-End Process Obfuscating Production Data into Test Environments?
Problem Given an organisations large scale dependency on use of production data for testing (due to complexities/nuances), a proven and trusted method of obfuscation, without compromising data integrity, is vital. How does iData’s obfuscation tool ...How Can iData Be Used To Provide Quality Assurance In A Data Migration Project?
Problem Data migration projects are often protracted and complex affairs. Comprehensive quality assurance activities are an essential part of such projects, in order to prove that the output is acceptable to all stake holders, and is thus fit for its ...