BitPeak at SQL Days 2024

Just like a year ago, we spent the last three days in Wroclaw at the annual SQLDay conference by Data Community Poland. Thoughts to remember from the 2024 edition:

💡 In his superb keynote speech, Chris Webb used explained what Microsoft Fabric is and how its new features relate to well-known software.

💡 During his presentation „Deciphering Data Architectures”, James Serra, discussed what Relational Data Warehouse and Data Lake are. We learned how they are structured, when each of them can be used or should not be used, and what should be considered when choosing the best suited option for your solution – because each architecture has it’s pros and cons. We talked about what a Data Mesh is, including how to decide what criteria it should fulfill, when implementation would be beneficial, and the challenges with developing and maintaining such an architecture.

💡 In Erwin de Kreuk 🚀 session, we discovered the details of OneLake and learned how it can centralize and manage data while leveraging the scalability, security, and advanced capabilities of Microsoft Fabric. The session highlighted why OneLake is a true game changer.

💡 Thanks to Benni De Jagere, we became familiar with the DirectLake concept and learned the essential details for connecting Power BI directly to OneLake. Understanding concepts like framing and warming up data is crucial for working effectively with this new connection.

💡 Adrian Chodkowski in „Microsoft Fabric – hidden germs” presented useful tools and functionalities that can accelerate and enhance work within Microsoft Fabric. We learned how to orchestrate multiple pipelines in notebooks and pass DAG definitions to them. There was a discussion on utilizing Spark Structured Streaming in notebooks. We learned how to create zero-copy clones in the warehouse and enable fast copy functionality in Dataflows Gen2 to enhance data ingestion.

Below you can see that we did a lot of learning, in a fantastic company!

Crowded networking area at SQL Days 2024