After upgrade to the latest version now this error id showing up **ArrowInvalid**

Using Streamlit
1

After upgrading to the latest streamlit version now when i run the function it display the below error:

ArrowInvalid: (‘Could not convert int64 with type numpy.dtype: did not recognize Python value type when inferring an Arrow data type’, ‘Conversion failed for column Data Type with type object’)
Traceback:
File “F:\AIenv\lib\site-packages\streamlit\script_runner.py”, line 350, in _run_script
exec(code, module.dict)
File “f:\AIenv\streamlit\app2.py”, line 1051, in
main()
File “f:\AIenv\streamlit\app2.py”, line 597, in main
explore(df)
File “f:\AIenv\streamlit\app2.py”, line 91, in explore
st.write(df_types)
File “F:\AIenv\lib\site-packages\streamlit\elements\write.py”, line 182, in write
self.dg.dataframe(arg)
File “F:\AIenv\lib\site-packages\streamlit\elements\dataframe_selector.py”, line 85, in dataframe
return self.dg._arrow_dataframe(data, width, height)
File “F:\AIenv\lib\site-packages\streamlit\elements\arrow.py”, line 82, in arrow_dataframe
marshall(proto, data, default_uuid)
File “F:\AIenv\lib\site-packages\streamlit\elements\arrow.py”, line 160, in marshall
proto.data = type_util.data_frame_to_bytes(df)
File “F:\AIenv\lib\site-packages\streamlit\type_util.py”, line 371, in data_frame_to_bytes
table = pa.Table.from_pandas(df)
File “pyarrow\table.pxi”, line 1479, in pyarrow.lib.Table.from_pandas
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 591, in dataframe_to_arrays
for c, f in zip(columns_to_convert, convert_fields)]
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 590, in
arrays = [convert_column(c, f)
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 577, in convert_column
raise e
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 571, in convert_column
result = pa.array(col, type=type, from_pandas=True, safe=safe)
File “pyarrow\array.pxi”, line 301, in pyarrow.lib.array
File “pyarrow\array.pxi”, line 83, in pyarrow.lib._ndarray_to_array
File “pyarrow\error.pxi”, line 84, in pyarrow.lib.check_status

code:

import streamlit as st
import pandas as pd

def explore(df):

# DATA

    st.write('Data:')

    st.write(df)

    # SUMMARY

    df_types = pd.DataFrame(df.dtypes, columns=['Data Type'])

    numerical_cols = df_types[~df_types['Data Type'].isin(['object',

                    'bool'])].index.values

    df_types['Count'] = df.count()

    df_types['Null Values'] = df.isnull().sum()

    df_types['Unique Values'] = df.nunique()

    df_types['Min'] = df[numerical_cols].min()

    df_types['Max'] = df[numerical_cols].max()

    df_types['Average'] = df[numerical_cols].mean()

    df_types['Median'] = df[numerical_cols].median()

    df_types['St. Dev.'] = df[numerical_cols].std()

    st.write('Summary:')

    st.write(df_types)

#show Summary

if st.checkbox("Show Summary"):

     explore(df)

what this error mean ? and How to fix this error

2

Hi!
I had the same problem. It’s a bug that came with streamlit 0.85.0. I hope developers will solve it soon.
The current solution is downgrade to version 0.84.2

pip install streamlit==0.84.2
3

Yes that is what i did i downgraded streamlit version to 0.84
Now there is a upgraded version 0.86 hope that this solve the problem.

4

Still having the same issue with latest release

5

Me too. Very simple read in of xlsx file with diverse column data types.

st.markdown("### Books In Print")

df = pd.read_excel("BIP4streamlit.xlsx")

df['title'].astype(str)

df['Contributor 1'].astype(str)

df

 File "/Users/fred/.virtualenvs/nimbleAI/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 571, in convert_column
    result = pa.array(col, type=type_, from_pandas=True, safe=safe)
  File "pyarrow/array.pxi", line 301, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('Could not convert A Day Book with Prompts with type str: tried to convert to int', 'Conversion failed for column title with type object')

upgraded to 0.88.

6

Hi all

A preferable solution for this is to use the old dataframe serializer by setting this in your .streamlit/config.toml file:

[global]
dataFrameSerialization = "legacy"

This allows you to continue upgrading to the latest version of Streamlit and getting all the latest goodies!

More info about Arrow here.

Meanwhile, I’ll forward this thread to our eng team. We want to find all instances where Arrow isn’t working well and fix them :slight_smile:

7 Likes
7

Hi there :wave:,

After some investigation, it turns out that pyarrow has an issue with numpy.dtype values (which df.dtypes returns).

The issue has been filed and hopefully will be taken care of soon.
In the meantime, a custom exception with a more useful error message will be thrown.

In some cases, a possible workaround is to convert DataFrame cells to strings with df.astype(str).
For example:

df = pd.DataFrame([["foo", "bar"]])
foo = df.dtypes.astype(str)
st.write(foo)

I hope this helps ease some of the burden.
Please let us know if that doesn’t work for you, and we’ll try to come up with a better solution.

And thanks for using Streamlit :guitar:

1 Like
8

Is it fixed in the latest release Streamlit 1.0??

9

@krishna The issue is still present in the latest PyArrow version (5.0 at the time of writing). Hopefully it will be taken care soon. The more useful error message is already available in 1.0.0.

2 Likes
10

A similar StreamlitAPIException is given when trying to print a dataframe with a type of psycopg2._range.NumericRange, a numeric range that comes from a postgres database. Pandas dataframes have no problem printing the results within a notebook but the error is given in streamlit.

11

I went back as you proposed. It worked!
thank you

12

Still this bug did not been solved in version 1.0
Hope streamlit creators take care of it in order to take benefetis of the version 1.0 features.

13

I am also experiencing this issue. Rolled back to .84 but I actually also need session states, so this kinda sucks.

14

Yes you’re right this is sucks because there are a lot of new features in the latest version that can be very useful.
Hope that this issue is being debugued by streamlit’s creators

1 Like
15

Dear users,

If the workaround proposed above didn’t solve your issue, please use the approach described here.

This allows you to continue upgrading to the latest version of Streamlit and getting all the latest goodies!

Sorry for the inconvenience! We’re waiting for Arrow to fix this issue.

16

I tried to add this line is the config.toml but when i tried to run streamlit app it crash and display the below error:

Traceback (most recent call last):
  File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "F:\AIenv\lib\site-packages\streamlit\__main__.py", line 23, in <module>
    main()
  File "F:\AIenv\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "F:\AIenv\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "F:\AIenv\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "F:\AIenv\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "F:\AIenv\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "F:\AIenv\lib\site-packages\streamlit\cli.py", line 161, in main_run
    bootstrap.load_config_options(flag_options=kwargs)
  File "F:\AIenv\lib\site-packages\streamlit\bootstrap.py", line 318, in load_config_options
    config.get_config_options(force_reparse=True, options_from_flags=options_from_flags)
  File "F:\AIenv\lib\site-packages\streamlit\config.py", line 1069, in get_config_options
    _update_config_with_toml(file_contents, filename)
  File "F:\AIenv\lib\site-packages\streamlit\config.py", line 946, in _update_config_with_toml
    parsed_config_file = toml.loads(raw_toml)
  File "F:\AIenv\lib\site-packages\toml\decoder.py", line 262, in loads
    original, i)
toml.decoder.TomlDecodeError: Found invalid character in key name: '#'. Try quoting the key name. (line 1 column 3 char 2)
ns\ms-python.python-2020.9.114305\pythonFiles\lib\python\debugpy\launcher' '58862' '--' '-m' 'streamlit' 'run' 'f:\AIenv\streamlit\app2.py' '--server.port' '5678'
Traceback (most recent call last):
  File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "F:\AIenv\lib\site-packages\streamlit\__main__.py", line 23, in <module>
    main()
  File "F:\AIenv\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "F:\AIenv\lib\site-packages\click\core.py", line 782, in main   
    rv = self.invoke(ctx)
  File "F:\AIenv\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    return ctx.invoke(self.callback, **ctx.params)
  File "F:\AIenv\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "F:\AIenv\lib\site-packages\streamlit\cli.py", line 161, in main_run
    bootstrap.load_config_options(flag_options=kwargs)
  File "F:\AIenv\lib\site-packages\streamlit\bootstrap.py", line 318, in load_config_options
    config.get_config_options(force_reparse=True, options_from_flags=options_from_flags)
  File "F:\AIenv\lib\site-packages\streamlit\config.py", line 1069, in get_config_options
    _update_config_with_toml(file_contents, filename)
  File "F:\AIenv\lib\site-packages\streamlit\config.py", line 946, in _update_config_with_toml
    parsed_config_file = toml.loads(raw_toml)
  File "F:\AIenv\lib\site-packages\toml\decoder.py", line 262, in loads
    original, i)
toml.decoder.TomlDecodeError: Found invalid character in key name: '#'. Try quoting the key name. (line 1 column 3 char 2)

and I cannot run the app unless i delete the config.toml file

so it looks that i am doing something wrong or this solution is not working

17

To: @leb_dev

It looks like it can’t parse the config.toml file.
Can you post the contents of the file? I’ll be happy to help you resolve it!

P.S. Or you can open an issue on GitHub if you like.

18

this is the content of the file without adding the line that you suggested before.

thank you for your time.

# Below are all the sections and options you can have in ~/.streamlit/config.toml.

[global]

# By default, Streamlit checks if the Python watchdog module is available and, if not, prints a warning asking for you to install it. The watchdog module is not required, but highly recommended. It improves Streamlit's ability to detect changes to files in your filesystem.
# If you'd like to turn off this warning, set this to True.
# Default: false
disableWatchdogWarning = false

# If True, will show a warning when you run a Streamlit-enabled script via "python my_script.py".
# Default: true
showWarningOnDirectExecution = true


[logger]

# Level of logging: 'error', 'warning', 'info', or 'debug'.
# Default: 'info'
level = "info"

# String format for logging messages. If logger.datetimeFormat is set, logger messages will default to `%(asctime)s.%(msecs)03d %(message)s`. See [Python's documentation](https://docs.python.org/2.6/library/logging.html#formatter-objects) for available attributes.
# Default: None
messageFormat = "%(asctime)s %(message)s"


[client]

# Whether to enable st.cache.
# Default: true
caching = true

# If false, makes your Streamlit script not draw to a Streamlit app.
# Default: true
displayEnabled = true

# Controls whether uncaught app exceptions are displayed in the browser. By default, this is set to True and Streamlit displays app exceptions and associated tracebacks in the browser.
# If set to False, an exception will result in a generic message being shown in the browser, and exceptions and tracebacks will be printed to the console only.
# Default: true
showErrorDetails = true


[runner]

# Allows you to type a variable or string by itself in a single line of Python code to write it to the app.
# Default: true
magicEnabled = true

# Install a Python tracer to allow you to stop or pause your script at any point and introspect it. As a side-effect, this slows down your script's execution.
# Default: false
installTracer = false

# Sets the MPLBACKEND environment variable to Agg inside Streamlit to prevent Python crashing.
# Default: true
fixMatplotlib = true

# Run the Python Garbage Collector after each script execution. This can help avoid excess memory use in Streamlit apps, but could introduce delay in rerunning the app script for high-memory-use applications.
# Default: true
postScriptGC = true


[server]

# List of folders that should not be watched for changes. This impacts both "Run on Save" and @st.cache.
# Relative paths will be taken as relative to the current working directory.
# Example: ['/home/user1/env', 'relative/path/to/folder']
# Default: []
folderWatchBlacklist = []

# Change the type of file watcher used by Streamlit, or turn it off completely.
# Allowed values: * "auto" : Streamlit will attempt to use the watchdog module, and falls back to polling if watchdog is not available. * "watchdog" : Force Streamlit to use the watchdog module. * "poll" : Force Streamlit to always use polling. * "none" : Streamlit will not watch files.
# Default: "auto"
fileWatcherType = "auto"

# Symmetric key used to produce signed cookies. If deploying on multiple replicas, this should be set to the same value across all replicas to ensure they all share the same secret.
# Default: randomly generated secret key.
cookieSecret = "c79e251310888b7b7e1ed34d0e3cc2c636ca779eff64b4effac90f27e329d7af"

# If false, will attempt to open a browser window on start.
# Default: false unless (1) we are on a Linux box where DISPLAY is unset, or (2) server.liveSave is set.
headless = false

# Automatically rerun script when the file is modified on disk.
# Default: false
runOnSave = false

# The address where the server will listen for client and browser connections. Use this if you want to bind the server to a specific address. If set, the server will only be accessible from this address, and not from any aliases (like localhost).
# Default: (unset)
#address =

# The port where the server will listen for browser connections.
# Default: 8501
port = 8501

# The base path for the URL where Streamlit should be served from.
# Default: ""
baseUrlPath = ""

# Enables support for Cross-Origin Request Sharing (CORS) protection, for added security.
# Due to conflicts between CORS and XSRF, if `server.enableXsrfProtection` is on and `server.enableCORS` is off at the same time, we will prioritize `server.enableXsrfProtection`.
# Default: true
enableCORS = true

# Enables support for Cross-Site Request Forgery (XSRF) protection, for added security.
# Due to conflicts between CORS and XSRF, if `server.enableXsrfProtection` is on and `server.enableCORS` is off at the same time, we will prioritize `server.enableXsrfProtection`.
# Default: true
enableXsrfProtection = true

# Max size, in megabytes, for files uploaded with the file_uploader.
# Default: 200
maxUploadSize = 200

# Enables support for websocket compression.
# Default: true
enableWebsocketCompression = true


[browser]

# Internet address where users should point their browsers in order to connect to the app. Can be IP address or DNS name and path.
# This is used to: - Set the correct URL for CORS and XSRF protection purposes. - Show the URL on the terminal - Open the browser - Tell the browser where to connect to the server when in liveSave mode.
# Default: 'localhost'
serverAddress = "localhost"

# Whether to send usage statistics to Streamlit.
# Default: true
gatherUsageStats = true

# Port where users should point their browsers in order to connect to the app.
# This is used to: - Set the correct URL for CORS and XSRF protection purposes. - Show the URL on the terminal - Open the browser - Tell the browser where to connect to the server when in liveSave mode.
# Default: whatever value is set in server.port.
serverPort = 8501


[mapbox]

# Configure Streamlit to use a custom Mapbox token for elements like st.pydeck_chart and st.map. To get a token for yourself, create an account at https://mapbox.com. It's free (for moderate usage levels)!
# Default: ""
token = ""


[deprecation]

# Set to false to disable the deprecation warning for the file uploader encoding.
# Default: true
showfileUploaderEncoding = true

# Set to false to disable the deprecation warning for using the global pyplot instance.
# Default: true
showPyplotGlobalUse = true


[s3]

# Name of the AWS S3 bucket to save apps.
# Default: (unset)
#bucket =

# URL root for external view of Streamlit apps.
# Default: (unset)
#url =

# Access key to write to the S3 bucket.
# Leave unset if you want to use an AWS profile.
# Default: (unset)
#accessKeyId =

# Secret access key to write to the S3 bucket.
# Leave unset if you want to use an AWS profile.
# Default: (unset)
#secretAccessKey =

# The "subdirectory" within the S3 bucket where to save apps.
# S3 calls paths "keys" which is why the keyPrefix is like a subdirectory. Use "" to mean the root directory.
# Default: ""
keyPrefix = ""

# AWS region where the bucket is located, e.g. "us-west-2".
# Default: (unset)
#region =

# AWS credentials profile to use.
# Leave unset to use your default profile.
# Default: (unset)
#profile =


[theme]

# The preset Streamlit theme that your custom theme inherits from. One of "light" or "dark".
#base =

# Primary accent color for interactive elements.
#primaryColor =

# Background color for the main content area.
#backgroundColor =

# Background color used for the sidebar and most interactive widgets.
#secondaryBackgroundColor =

# Color used for almost all text.
#textColor =

# Font family for all text in the app, except code blocks. One of "sans serif", "serif", or "monospace".
#font =
19

I don’t get any errors when I use that TOML file. Can you please post the version of the config.toml that produces the error?

Thank you!

20

I did not understand what you mean by the version of config.toml.

Each time i try to create the file by typing this statement streamlit config show > ~/.streamlit/config.toml than try to run the app the system crash and display the error that i showed above.
Based on this post everyone has delete the config file and everything work again.

How to fix this because i am stuck in the old version of streamlit. v 0.84