Parquet vs Feather

Hey guys, I am working on a project that has a dataset with more than 200 million rows, and I would like to use it efficiently. My first question is, since I will be using a S3 store from AWS, which file is the best to handle this amount of data? My second question is, since st.connection does not support โ€œfeatherโ€ files, this means that I should use parquet instead or it doesnt matter?

Thanks in advance.

Hi @caiosoter

Perhaps these 2 related blogs may provide you some insights on the use of data in the building of a performant app:

Hope this helps!