Data engineering helps make data more useful and accessible for consumers of data. To do so, ata engineering must source, transform and analyze data from each system. For example, data stored in a relational database is managed as tables, like a Microsoft Excel spreadsheet.

We are happy to announce Data Engineering with real-time use case Knowledge Sharing sessions

📑 Syllabus

🎯 𝐏𝐚𝐫𝐭-𝟏:𝐔𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 𝐒𝐨𝐮𝐫𝐜𝐞 𝐒𝐲𝐬𝐭𝐞𝐦

📌 Have taken Retail Banking as case study and made it easy to understand for beginners and intermediate level. 📌 Why? This will in-turn ensure, have good clarity while providing solution and designing

🎯 𝐏𝐚𝐫𝐭-𝟐:𝐑𝐞𝐚𝐥 𝐰𝐨𝐫𝐥𝐝 𝐝𝐚𝐭𝐚 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 𝐜𝐚𝐬𝐞 𝐬𝐭𝐮𝐝𝐢𝐞𝐬

📌 Requirement - Will define case study, provide scenario-base problem 📌 Why? This will help us to focus on architecture, design and provide solution

🎯 𝐏𝐚𝐫𝐭-𝟑:𝐎𝐯𝐞𝐫𝐯𝐢𝐞𝐰 𝐨𝐟 𝐝𝐚𝐭𝐚 𝐦𝐨𝐝𝐞𝐥𝐢𝐧𝐠

📌 Design simple data warehouse with dimension and facts 📌 Design Data Lake 📌 Azure SQL and SQL Server

🎯 𝐏𝐚𝐫𝐭-𝟒:𝐃𝐚𝐭𝐚 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧

📌 Batch and Real time streaming - ingestion, transformation, delivery and movement of data 📌 Batch includes Azure Data Factory 📌 Real Time streaming includes - Azure streaming 📌 Data Lake ingestion includes - Data Bricks 📌 Walk though on Kafka

🎯 𝐏𝐚𝐫𝐭-𝟓:𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 𝐜𝐨𝐧𝐭𝐫𝐨𝐥 𝐚𝐧𝐝 𝐃𝐚𝐭𝐚 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞

📌 Error handling 📌 Monitoring and control 📌 Testing 📌 Data Quality and Data Governance - Tools in market

🎯 𝐏𝐚𝐫𝐭-𝟔:𝐕𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧

📌 Do we have all the data? in desired quality? 📌 Visualization using Power BI 📌 Understand value to business, customer related service and decision making

Loved it?

𝐇𝐢𝐭 👍 𝐬𝐡𝐨𝐰 ❤️ 𝐓𝐚𝐠 𝐘𝐨𝐮𝐫 𝐟𝐫𝐢𝐞𝐧𝐝 ❤️

Community and Social Footprints :

• Srinath Babu
• GitHub
• Twitter
• YouTube Cloud DevOps Free Trainings
• Linkedin Page
• Linkedin Group
• Discord Channel
• Dev