Different results in local streamlit and deployed streamlit app

Dear streamlit community,

while working on a rather big app, I found a weird deviation from what my local code / local run of the streamlit app produces, compared to what the deployed app produces.
In order to get help, I isolated the code snippet that is deviating and deployed the app and I still get the same issue: different results in deployed app and local app/code.

The code loads a pickle file from the repo and builds, then fits a VAR model to find the best model order.

Locally, my best model order is calculated as 8, while in the deployed app, the best model order is calculated as 9. from the resulting table, it is clear that the resulting KPIs differ as of line 7.

See screenshots here:
Local app:
image

Deployed app:
image

I made sure to locally work in a virtual environment and used the same requirements.txt file for dependencies, double checked and they are the same in the deployed app and in my virtual environment.

The deployed app is to be found here: https://app-var-test.streamlit.app/

I work with Windows and Pycharm locally, but tested the code in Spyder and terminal as well, all resulting in 8 as best model order.

Here is the code for the Var model:

import pickle
from statsmodels.tsa.vector_ar.var_model import VAR
import numpy as np
import pandas as pd
import streamlit as st

with open("data_in.pickle", 'rb') as input:
    df = pickle.load(input)

model = VAR(df, freq='MS')
aic, bic = [], []
o_range = np.arange(1, 11)
for i in o_range:
    result = model.fit(i)
    aic.append(result.aic)
    bic.append(result.bic)

lags_metrics_df = pd.DataFrame({'AIC': aic,
                                'BIC': bic},
                               index=range(1, len(aic) + 1))

best_model_order = lags_metrics_df.idxmin(axis=0)['AIC']
print(best_model_order)
print(lags_metrics_df)

Requirements:

streamlit===1.33.0
numpy==1.26.4
pandas==2.2.1
statsmodels==0.14.1

Any help for this is highly appreciated (it’s this sort of error where you think you are turning crazy :slight_smile: )

Thank you!
Julia

update: I tried the same with updated versions of the necessary modules in a different app and run into the same issue.

streamlit===1.38.0
statsmodels==0.14.2

`

I was able to reproduce the issue, fitting the model gives different results in both environments. Maybe the different operating systems are causing this? I will try a Linux system later if I have a chance.

1 Like

I somehow suspect the same. Will also ask in the statsmodels gitpage.
One thing I did not check before were different python versions, but I set up a copy of the app with python version 3.11 and still have the same issue.
I can’t replicate this in Linux myself so that would be really great to test, thanks!

That’s interesting. Tried on Ubuntu 20.04 with Python 3.12, 3.11 and 3.9, statmodels 0.14.2 and 0.13.0, numpy 1.26.4 and 2.1.0… results were all consistent to the deployed version.

But on Windows (Python 3.9) results differ:

That’d be better, I doubt this is caused by Streamlit.

Explained here:

1 Like

Yes, statsmodels forum provided an answer.
Thanks for your help here!

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.