NotImplementedError: Cannot copy out of meta tensor; no data! when i am deploying my ML app on streamlit i am facing this issue

LLMs and AI
1

I am getting this error when i am deploying my LLM on streamlit application i have whole code in 1 main.py file which i am running by streamlit and this main.py contain all my LLM and streamlit interfaces.

2

Hi @chirag_aswal,

Thank you for sharing your question with the community!

Your post is missing a code snippet and a link to your app’s GitHub repo. Please check out our guidelines on how to post an effective question here and update your post to help the community answer your question.

3

Encountering the same issue here on an intermittent basis. The app is hosted on community cloud in a private repo. No error when using the same search criteria locally.

App: Japanese company disclosure search app using sentence-transformers + typesense

Error message:

NotImplementedError: Cannot copy out of meta tensor; no data! Please use
torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module
from meta to a different device.

  1. Code to load vector model
    device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)
    torch.classes.path =
    model_path = ‘model’
    model = SentenceTransformer(model_path, device=device)

  2. Requirements
    streamlit==1.45.1
    typesense==0.21.0
    sentence-transformers==4.1.0
    torch==2.7.0
    torchvision==0.22.0

  3. Extract from logs
    /home/adminuser/venv/lib/python3.12/site-packages/torch/nn/modules/module.py
    :1348 in convert

    1345 │ │ │ │ )
    1346 │ │ │ except NotImplementedError as e:
    1347 │ │ │ │ if str(e) == “Cannot copy out of meta tensor; no data
    ❱ 1348 │ │ │ │ │ raise NotImplementedError(
    1349 │ │ │ │ │ │ f”{e} Please use torch.nn.Module.to_empty() i
    1350 │ │ │ │ │ │ f"when moving module from meta to a different
    1351 │ │ │ │ │ ) from None