Marcelo Rabello RossiinTowards Data ScienceLeveraging Apache Spark to execute billions of operations on AWS S3It is just moving files, not rocket science, right? Until you have to do it for over a million files and tens of terabytes…6 min read·Dec 10, 2020--3--3
Marcelo Rabello RossiinTowards Data ScienceA journey to Airflow on Kubernetes… or how I got it to work, piece by piece, in a logical way12 min read·Jul 20, 2020--4--4