How to Schedule a task

Hello guys

I have deployed an application, which provides a data analysis from a CSV file. Howver, everyday it gets more data, and the website gets slower for the user.

Then, I’d like to know if is it possible to schedule a background task to download daily this CSV file to the application, to make faster for the user.

We started a thread to discuss these kind of engineering tasks at Data engineering best practices

One quick thought, but an ever growing csv that is loaded into memory is not a scalable solution. Ideally you would load your data into a true database and query that

1 Like