Getting everyone on the same page when it comes to data is an incredibly difficult endeavor.
The problem with data & teams
Databases are your businesses source of truth. When you have a question about what is happening with any particular metric, in some form you are most likely interfacing with objects stored on your database to retrieve the answers you seek. With this as a fact, making certain your entire organization is using the same data language is of increasing importance.
In reality, however, most organizations do not talk in the same language when it comes to data. Your frontend team doesn't always know exactly what the database itself stores. Your product team talks in goals. Your business team talks in metrics extracted, transformed, and loaded somewhere else. Getting everyone on the same page when it comes to data is an incredibly difficult endeavor.
How do we solve it?
Solving for cross-organization communication is not a task done in a day. Instead you make a committed effort for it to be done, and take it one team at a time. What better spot to start than where your data actually lives – in your database. Challenge your team to plan, discuss, and think through how each new feature should be structured in your database(s) so it serves as the foundational groundwork for the entire company to talk in the same language as it is stored in. User objects are in an account table, sessions in a session table, therein and so forth. I don't for a minute take for granted how difficult this could be especially if you already have data tables in place. Cleaning existing structures into more sensical talking points that fit the business model of today can become quite the challenge but we won't cover that in this blog post.
So, how should this problem be solved? We now have established the source of trust and truth will stem from our database, it's now turn to making other teams privy of our new global data structures. Enter, database introspection.
What is Introspection?
Database introspection is the ability to evaluate your schema, tables, columns, foreign keys, indexes and more within a database to establish what data structures exist, their respective data types, and how – if at all – they are interconnected. For example, if you have a table in your database named account with a handful of columns defining attributes of a particular user in a given row then you can construct sensible data models that can persist outside of your database.
Power stems from your organizations ability to do this with your databases, not only because it enables data models to be extracted from the database and established in other projects, but by doing this that common data language can be spread maintaining our goal of a singular source of truth.
Automatically Sync Data Models
At this point we've hammered away at how important it is for you to construct a universal data language across the entire company. Now it's time to put it into action.
Keeping your database structure in sync with your application layers (backend, frontend, etc) can be quite tedious and tiresome to do manually and on an ad-hoc basis as product requirements see fit. Instead of that we are going to show you how to use the Outerbase Query Builder open-source library (npm package here) to have a single command in your project that will sync data models from your database to your project in seconds.
Step 1: Link your database in Outerbase
If you don't already have an Outerbase account, it's quick to setup. Outerbase serves as the ultimate database collaboration tool for developers and teams.
Whether you have your database connection strings, or a login into your database provider such as Neon, Turso, or DigitalOcean – you can follow the guide below to get your first database connected:
Step 2: Grab your API key
From your Outerbase workspace home page you should see a list of bases as cards, each card representing one database.
Click on the database you wish to sync data models with
On the far bottom left sidebar, find the Settings cog icon and click it
From the Base Settings sidebar click "General"
Now on the right hand side find the "API Key" section
If no key exists, click "Generate API Key"
Copy your API key
Step 3: Install the NPM package
Now we are ready to include the Query Builder library from Outerbase into our projects. With this package comes the ability to do database introspection and generate Typescript models automatically.
Inside your project within Terminal run the following command:
npm i @outerbase/query-builder
Step 4: Add script to the package.json
Once the NPM package has been installed you can look at your package.json file and confirm that @outerbase/query-builder is now listed as a dependency. Now we can utilize the sync-database-models command it exports for us to generate our models by providing two property variables:
PATH= from the source of your project, where should the models be generated and storedAPI_KEY= paste in your Outerbase API key you obtained from step 2 above
"scripts": {
"sync-models": "sync-database-models PATH=./folder/path/to/add/models API_KEY=outerbase_api_key"
}
Step 5: Sync your data models
Everything is in place. Our database has been added to Outerbase, which will serve as the place of origin on where the database introspection occurs. It will iterate over your schemas, tables, columns and more to help define all of the Typescript models your frontend should contain to be in sync with your database.
Running the following command will automatically trigger the aforementioned work and you should see the fruits of your labor exported wherever you defined your PATH to be:
npm run sync-models
Any time your database changes your projects should run the above command. This will help your application at all layers be in sync with the source of truth.
Summary
We've covered a fair bit in here with some important topics.
Establish a company wide data language
Use your databases as a source of truth for establishing that language
Sync your data models with your projects
Enjoy co-workers saying the right words for the right things
If you enjoyed the topics covered in this blog post then there is a high chance you are also looking for a tool such as Outerbase. With Outerbase we give you and your team an entire platform to all share the opportunity to interact on your data as if it were the source of truth by empowering users to safely query, visualize with dashboards, and hold conversations with AI. Sign up today & invite your entire team!
