There are some low-code ETL tools, but is that enough? Companies want to treat their data pipelines like mission-critical apps. They want DevOps for data, with GIT, test coverage, and CI/CD. In addition, companies need end-to-end visibility of their data pipelines, including monitoring, metadata, and column-level lineage.
Where are the data tools for these capabilities and how do non-programmers use these?
In today’s podcast, you’ll see how Prophecy’s low-code data engineering platform, built on Spark and Airflow, provides a complete solution for developing, deploying, and monitoring data pipelines. We’ll speak with Raj Bains, who is the founder & CEO of Prophecy.
Raj has spent two decades building powerful tools – Microsoft Visual Studio, founding engineer for CUDA at NVIDIA, product manager for Hive at Hortonworks. He is passionate about making all organizations productive with data and making the lives of data engineers better by reducing toil and increasing the joy.
Also, if you like what you hear, Prophecy has a special offer for our listeners. Go to Prophecy.IO/request-a-demo and put in “SED” for the referral code. If you decide to purchase within 3 months of the demo, you get 10% off. If you purchase after 3 months, you still get 5% off.
Sponsorship inquiries: sponsor@softwareengineeringdaily.com
Transcript
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com to get 15% off the first three months of audio editing and transcription services with code: SED. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Sponsors
WorkOS is a developer platform to make your app enterprise-ready. With a few simple APIs, you can immediately add common enterprise features like Single Sign-On, SAML, SCIM user provisioning, and more. Developers will find beautiful docs and SDKs that make integration a breeze. WorkOS is kind of like “Stripe for enterprise features.” WorkOS powers apps like Webflow, Hopin, Vercel, and more than 100 others. The platform is rock solid, fully SOC-2 compliant, and ready for even the largest enterprise environments. So what are you waiting for? Integrate WorkOS today and make your app enterprise-ready. To learn more and get started, go to softwareengineeringdaily.com/workos
Cox Automotive transforms the way that the world buys, sells, owns, and uses cars – and they have the data to understand how the world gets from point A to point B. With brands like Kelley Blue Book, Autotrader, Dealer.com and others, Cox Automotive is hiring software engineers, data scientists, scrum masters, and other technologists to help create meaningful change in the industry. If you want to innovate in a collaborative workplace, one that values your time and work-life balance, then visit COXAUTOTECH.COM to find career opportunities at Cox.
Puppet, the industry leader for infrastructure automation, invites you to a fireside chat to learn how to increase collaboration and build a culture of joint accountability for compliance and security across your organization.
Join Alex Hin, Senior Product Manager at Puppet, for a discussion of how IT, security, compliance, and risk management teams can work together to protect your organization from risk.
You can join Puppet for the conversation on Thursday, April 7th at 11 AM eastern and start building your own culture of accountability that enhances compliance by registering at events.puppet.com/fireside.
strongDM lets you manage and audit access to servers, databases, and Kubernetes clusters, no matter where your employees are. With strongDM, you can easily extend your identity provider to manage infrastructure access. You can automate onboarding, offboarding, and moving people within roles. strongDM. Manage and audit remote access to infrastructure. Start your free 14 day trial today at: strongdm.com/SEDaily
Data engineers struggling with unreliable data rely on Monte Carlo, the world’s first end-to-end, fully automated Data Observability Platform! Monte Carlo enables data teams with visibility into the quality and reliability of their analytical data to maximize business impact. Start trusting your data with Monte Carlo today! Visit softwareengineeringdaily.com/montecarlodata to learn more.
