Learning Data Engineering in 2023
tronghuy2807What core skills do you need to succeed as a Data Engineer? What is the state of data engineering today? In 2023 the demand for data engineers is showing no…
Understanding Database Types
The success of a software application often hinges on the choice of the right databases. As developers, we're faced with a vast array of database options. It is crucial for…
Installing Apache Airflow on AWS EC2
Table of Contents Setting up EC2 on AWS Python and Pip Creating a virtual environment Installing Airflow Initialising Airflow source: https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+logos Apache Airflow is a widely used open source tool in…
Data Engineering Design Principles You Should Follow
Design principles to follow that will take your data engineering skills to the next level. Data pipelines don’t need to be too complex. Software engineering is far more mature than…
Why ChatGPT Won’t Replace Data Engineers
Table of Contents Utilising ChatGPT ChatGPT’s Limitations Closing Thoughts It seems that we are living in a time that will one day be recognised as the beginning of the AI…