Getting Started with OvalEdge Data Catalog

This guide helps you to navigate through various features and functionality of OvalEdge. It will help you to get started with OvalEdge quickly.

Basic Navigation

The first step is the user login. Use your username and password to log in. Press on the Forgot my Password button to reset your password. Once you log in, you will see a left menu and a top navigation bar.

Left Menu
The Left Menu provides access to various applications within OvalEdge.
You know that you can press on the right arrow to expand and collapse the left menu.

Top Menu bar
The Top Menu is used for searching or using as a gateway to favorite applications.

Bookmark
You can bookmark your favorite application, table, or report by navigating to that page and clicking the bookmark icon.
You can pin your favorite bookmarks on the browser header and use it as an extended favorites bar for easy navigation.

Quick Search Bar
You can search for any table, column, file, or report using the quick search bar. In case the results are not available, you can perform a detailed search using Smart-Catalog application.
Adding a profile picture is recommended because it provides three times more visibility about you in your company.

Profile
This is a basic application for you to update your password, profile picture, or phone number.

Messages
The Message application displays all of your messages. The messages displayed in bold require your attention the most as they are specifically directed towards you.
You can press a unique identifier to sign in your messages to get to the entire data catalog.

Dashboard
The Dashboard application visually provides you all of the insights about your company.

Navigate / Research / Explore / Find Data

Navigating Data Functionally (Tags)

Use the tag application to get to the data quickly. Simply keep clicking the subject area you are interested in and keep exploring. Tag applications have a tree structure. Once the data is displayed, you would see the statistics and all the data objects associated with it.

Navigating / Research /Explore Data Technically (Smart Catalog)

Use Smart Catalog application to navigate to the data using databases, schema, files, infographics like Tableau, Power BI reports. You can use various ways to research data. Either you can type the keyword in Explore All or research based on the specifics tabs.

Infographics application is where you can search all the Tableau, Power BI, or any other kind of reports, charts, etc.

Data Story application is the entire knowledge base for your data. You can create wiki kind of documentation in data story.

Changes let you explore all the changes in data; using this, you can research what data element has changed and compared by date.

Metadata lets you understand the latest changes in the metadata. Simply sort by table change date or column change date to understand the change and click on the table to compare the metadata changes.

Navigating / Explore Data using Glossary of Terms

When you implement glossary of terms, you can also explore data using business terms. Click on Glossary of Terms Application and search your desired business term. When the business term is found, you can read more about the terms, its usage, and all the details and also see its related terms.
You can also view associated data dictionary.

Understanding Data

Once you click on a table, file, infographics, or query, you can see many details about the specific data object and its relationship with other objects. OvalEdge provides much information about a selected object by which you can quickly understand the data using various data statistics. OvalEdge also provides a comprehensive view of all kinds of relationships like references, entity-relationship, pattern relationship, etc.

• When you are writing content, you can click on @ sign to add links to other data objects.

• Whenever you see blue color text or label, this means you can click on it to understand more.

• You can reach to help by just hover and click on the question mark.

Statistics / Profiled Results

Profiled results provide a way to understand data quickly, and you can see various stats like top 50 values, various counts of every field.

Top Users

When you have any question about the data, you can see the insights who are the right user who can answer your question.

Endorsements

When you are using some report, or table in your analysis, you should endorse that table, so others can see various endorsements.

Entity Relationships

Entity relationships are direct relationships between various tables. These relationships are identified by two scores, Similarity Score and Join Score. When Similarity score is trending towards 100, this means these two tables are duplicate, and when Join score is trending towards 100, this means that its a perfect primary key and foreign key relationship.

Lineage

Using lineage tab, you can understand where the data is coming from and how it’s processed and where it’s going. OvalEdge also shows the lineage at the column level by clicking the circle icon between two main objects.

References

When the table is used by any other object in documentation, query, it’s showing in references.

Collect Data specific to your project

When you are working for a specific project and want to collect data from the entire organization, you can use Project application. You can create and share the project with your colleagues. Make a project as your default project. Now anywhere in your OvalEdge data catalog, you can click on Project icon to add the data to your project. You can also add comments specific to your project. This way, all the data objects are assembled in your project.

Collaborate / Ask questions about data

OvalEdge allows you to ask questions about the data from anyone in the organization. However, you would like to ask a question from either its designated steward, owner, or a user who is using that data object. Using OvalEgde, you can understand the users who are mostly using that data object. It provides three number, Views, Queries, and Questions responses.
 To ask a question, simply click on the user and type your question. You can add more users or data objects by a @ sign.

Analyze / Query Data

When profiled results are not sufficient to give you detailed information about the data and your data related question is not answered, use the Query Sheet to either build a query or write a query. OvalEdge allows non-SQL users to write SQL using step by step.

Building OvalEdge data catalog

This guide for the users who want to connect OvalEdge to multiple sources and document/catalog the data into OvalEdge so that all the end users can find and explore data with ease.

The building of OvalEdge data catalog is done primarily two ways:

• Automatic
• Human curation with AI assistance

Automatic by various OvalEdge integrations and algorithms

OvalEdge has a variety of algorithms to build the data catalog. You would need administrative access to build the data catalog.

Crawling

Crawling allows OvalEdge to connect to various applications, databases, data warehouses, data lake, reporting & visualization technologies. Crawling brings all the metadata like tables, columns, data types from source systems into OvalEdge. Crawling in OvalEdge is as simple as connecting.  Use appropriate technology to connect to the database and then click the crawl button to crawl the desired database.

Note: To connect the desired database, you may have to add the JDBC jar file, into OvalEdge jar folder which is configured in configurations.

Profiling

Profiling collects various statistics about the data. OvalEdge has pre-built algorithms and queries which execute on the database server. OvalEdge can profile various relational databases, but can not profile visualization software and data lake natively.

File system like NFS, S3, HDFS does not have the processing power, so OvalEdge uses its proprietary technology to perform the profiling.

Profiling Controls

As OvalEdge uses the database’s processing power to profile/analyze the data so it may create a certain workload on databases. To control this workload, OvalEdge provides profiling settings. Configure these settings appropriately based on the database server’s processing power and after discussion with DBAs. OvalEdge default settings can be used for playing it safe.

Relationship Building

OvalEdge support relationship building via the various mechanism.

Relationship via Crawling
When there are primary, foreign keys are defined in the database; OvalEdge automatically picks as a relationship while crawling.

Pattern Relationship via Profiling
When we profile a database, all the relationship which have similar pattern gets identified.

Relationship building for Hadoop based  data lake only
You can build various relationships using Hadoop based algorithms.

Lineage Building

OvalEdge has a comprehensive framework to build lineage. Lineage building is performed using OvalEdge advance job framework. Select the appropriate job to build the lineage.

Human Curation with AI Assistant

When data is well organized, it’s easy for end-users to navigate and research the data. Use tags application to navigate and organize the data. In tag application, you can use + icon to create new tags and drag the tag to make its child. Use edit icon for adding a better description and use Recommendation for AI-assisted tagging application.

Security & Data Governance using OvalEdge

Administrators can provide access to data and metadata by various roles are permissions and can also assign owner, stewards for various data objects.

Data Owner means that they are responsible for controlling the access and writing policies about the data access. They would also be the approver of any workflow request to grant access to data.

Data Steward is responsible for managing metadata, business glossary, and assigned data objects. They would also be the approver of any changes in the metadata and tags.

Roles may be a business function or group of people require certain data access. A user can be part of multiple roles.

Permissions are the type of access any data object will have to a given role. OvalEdge has multiple kinds of permissions.

Meta Read makes data accessible and lets you read the metadata and descriptions.

Meta Write lets you update various descriptions.

Data No Access means that although they can see the metadata but can not view the data.

Data Preview means that the user can view the data, can see profiled results but can not download or query the data.

Data Read allows the user to view as well as download and query the data.

Data Write allows the user to edit or update the data. In order to provide the Data Write access, the service account of that database must have Data write permissions.

Admin allows providing access to others via workflow.

OE_ADMIN is the global administrator and allows users to configure the security and every possible function in OvalEdge.

A domain is a high-level function of any organization. Generally, Site, Business Unit are domains.

Data Asset Group (DAG) is a group of Data Assets. Any table, file, or infographics must belong to a DAG. Assignment of the owner, steward, and security groups are at the DAG level. By default, all the tables belong to a schema; Infographics/reports belong to Info Groups and files belong to a folder. Schema, Info group, and folder are also a DAG.  You can also create a DAG Tag so that you can change the default behavior of table/file/infographics to DAG tag level.

Glossary of terms is an application where all the business terms can be maintained. In OvalEdge these terms are also a tag and can be applied to a table or field so that it becomes a data dictionary.

Note: You can load these terms from excel file.