Comment by ensemblehq

Through many different companies, we’ve primarily handled this with validation/transformation scripts across a variety of tools. What’s useful is attaching observability/logging and having each transformation step do one thing only (borrowing from the Unix philosophy). Having a good understanding of the schema designs for each system is key - at some organizations, we’ve codified the schemas. Pydantic models (and similar) have been very useful as well.