This was a complicated feature. When you have a system that manages dozens or even hundreds of databases, you need to handle “governance” of the data. This means you need to have consistent header names, consistent metadata, easy transformations, lineage, and more.

The design itself was fairly straightforward. You had classes of tables and meta data on columns. Apply it to the database and voila instant consistency.

I provided the original design and architecture and quickly handed it off to Ziyi Yue and Alexis Zheng. They did an amazing job with it, iterating and finishing the design.

I am not showing all of the interactions of course. Feel free to contact me for a more detailed walk through.