Upload Zip - Open Shapefile within in Memory

Long-Version
I am refactoring an application from a desktop GUI over to streamlit to be deployed online and I’m coming across an issue that I can’t seem to get my head around. While working on the desktop version of the app, it’s easy to manipulate files on the local machine without issue Share a clear and concise description of the issue. Aim for 2-3 sentences; but for a webapp I need to be able to get the data uploaded and then work on the contained files. So I want to be able to handle a zip file that contains the output from another program (ESRI Shapefiles and Excel files are the ones I’m interested in) do some work with those files and then produce a download that contains the updated files. This is causing me some headaches when I try to do this without extracting the zip file first

Short-Version
I need to open shapefiles and excel files that match certain filenames from within a zip file uploaded to the streamlit app by a user without extracting it first. Expected input is a zip file containing flat structure with various files including shapefiles and excel files (I haven’t got round to using openpyxl yet)

Code

import io
import zipfile
import streamlit as st
from io import BytesIO
import pandas as pd
import geopandas as gpd
from fiona.io import ZipMemoryFile


def read_shapefile_from_zip(zip_file, shapefile_to_open):
    with zipfile.ZipFile(zip_file, 'r') as my_zip:
        for file_list in my_zip.namelist():
            if file_list == shapefile_to_open:
                st.text(f'match {file_list}')
                with my_zip.open(file_list, "r") as data:
                    data = io.BytesIO(data.read())
                    st.text(data)
                    df = gpd.read_file(data)
                    st.dataframe(df)
                    
                    
shape_file = 'data1.shp'

file_uploaded = st.file_uploader(label='Please choose zip file to upload:', type='zip', key='zip_file')
if st.button(label='Run', key='run'):
    read_shapefile_from_zip(file_uploaded, shape_file)

Iv’e also tried this and a few others trying to make this work the way I want

    df = gpd.read_file(f'zip:///{ZipMemoryFile.open(zip_file)}!{shapefile_to_open}')
    st.dataframe(df)

Expecting the zip file to be read and the data dropped into a dataframe (st.dataframe in the function is there to see if it worked)

Actual behavior:

Error on main code is as follows

DriverError: ‘/vsimem/7d7e6cff88ab42ab8334824e09374ff9’ not recognized as a supported file format.

Traceback:

File "/home/richard/PycharmProjects/graph_node_ordering/venv/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)File "/home/richard/PycharmProjects/graph_node_ordering/testing.py", line 87, in <module>
    read_shapefile_from_zip(file_uploaded, shape_file)File "/home/richard/PycharmProjects/graph_node_ordering/testing.py", line 64, in read_shapefile_from_zip
    df = gpd.read_file(data)
         ^^^^^^^^^^^^^^^^^^^File "/home/richard/PycharmProjects/graph_node_ordering/venv/lib/python3.11/site-packages/geopandas/io/file.py", line 281, in _read_file
    return _read_file_fiona(
           ^^^^^^^^^^^^^^^^^File "/home/richard/PycharmProjects/graph_node_ordering/venv/lib/python3.11/site-packages/geopandas/io/file.py", line 322, in _read_file_fiona
    with reader(path_or_bytes, **kwargs) as features:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/home/richard/PycharmProjects/graph_node_ordering/venv/lib/python3.11/site-packages/fiona/collection.py", line 783, in __init__
    super().__init__(self.virtual_file, vsi=filetype, **kwds)File "/home/richard/PycharmProjects/graph_node_ordering/venv/lib/python3.11/site-packages/fiona/collection.py", line 243, in __init__
    self.session.start(self, **kwargs)File "fiona/ogrext.pyx", line 588, in fiona.ogrext.Session.startFile "fiona/ogrext.pyx", line 143, in fiona.ogrext.gdal_open_vector

So anyone have experience with this sort of thing that can help me get to the bottom of this before I just extract the files and delete after processing?

Hi @Richard_Dean

There seems to be a similar issue on Stack Overflow that may provide some directions to look into.

Also, have you tried unzipping the files and then working on the target file instead?

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.