Great! You have an empty list so with the length of the list being 0, there were no valid selections that could be made. So whenever you need to look up from lists, make sure to include checks that the lists are non-empty and that you don’t exceed their size.
So I realized I didn’t update the code I was running to the github folder so the program avidanwordembeddings was not what I was running on my machine and now I think this brings in a new problem because the part of the code that saves post_list_sample and final_weibos is overwriting those respective csv files and when the code runs its says there is nothing in those files to reference. I had commented out all the lines of code that were under the header Save post_list_sample to a csv and Save final_weibos to a csv because I was noticing that I think this part of the code was overwriting the data from the part that has the header takes 30-45 mins to run and I am getting this error now.
EmptyDataError: No columns to parse from file
Traceback:
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py”, line 561, in _run_script
self._session_state.on_script_will_rerun(rerun_data.widget_states)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\safe_session_state.py”, line 72, in on_script_will_rerun
self._state.on_script_will_rerun(latest_widget_states)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 543, in on_script_will_rerun
self._call_callbacks()
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 556, in _call_callbacks
self._new_widget_state.call_callback(wid)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 277, in call_callback
callback(*args, **kwargs)
File “C:\Users\avida\FreeWeibo\avidanwordembeddings2.py”, line 371, in WEBIO
post_list_sample_new = pd.read_csv(“C:\Users\avida\FreeWeibo\post_list_sample.csv”)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\util_decorators.py”, line 211, in wrapper
return func(*args, **kwargs)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\util_decorators.py”, line 331, in wrapper
return func(*args, **kwargs)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\io\parsers\readers.py”, line 950, in read_csv
return _read(filepath_or_buffer, kwds)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\io\parsers\readers.py”, line 605, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\io\parsers\readers.py”, line 1442, in init
self._engine = self._make_engine(f, self.engine)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\io\parsers\readers.py”, line 1753, in _make_engine
return mapping[engine](f, **self.options)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py”, line 79, in init
self._reader = parsers.TextReader(src, **kwds)
File “pandas_libs\parsers.pyx”, line 554, in pandas._libs.parsers.TextReader.__cin
I have updated the github file to resemble exactly the file I am referencing for the word embedding part of the project.
For clarification those lines of code that save to csv are designed to allow for me to essentially skip the code that says it will take 30-45 mins to run and save that time but something is wrong right now where the files are being overwritten somehow and then when it gets to the lines where it references to those csv files it says the error above.
Unfortunately, it’s just too imcomplete for me to run. That feeweibo-09-15-2022.csv just isn’t readable, even though I tied installing all the chinese character sets I could. Without that file, too many things are empty and I keep having to code around lines to check if results are non-empty. Sorry, but I don’t think I’ll be able to debug it…
1 Like
Ok no problem I think I will just have the majority of the program that wasn’t working in the Streamlit app run in the backend have the chart display. I and you have spent waaaaay to much time trying to remedy this and I can’t thank you enough!
2 Likes
Hey @mathcatsand question for you I am trying a work around for my code that we were working on I split the code avidanwordembeddings into two and then I am having a button reference that second files. I am getting an error called KeyError: 1 for the below line of code can you help me figure out why this is happening?
temp_list = []
num_weibos = 10
sim_matrix = [0]*num_weibos
#print(sim_matrix)
for i in range(num_weibos):
for j in range(num_weibos):
sim_i_j = model.n_similarity(final_weibos[i], final_weibos[j])
temp_list.append(sim_i_j)
sim_matrix[i] = list(temp_list)
temp_list = []
#print(len(sim_matrix))
print(sim_matrix)
print(max(sim_matrix))
Here is the full error message
Blockquote
'KeyError: 1
Traceback:
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py”, line 561, in _run_script
self._session_state.on_script_will_rerun(rerun_data.widget_states)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\safe_session_state.py”, line 72, in on_script_will_rerun
self._state.on_script_will_rerun(latest_widget_states)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 543, in on_script_will_rerun
self._call_callbacks()
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 556, in _call_callbacks
self._new_widget_state.call_callback(wid)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py”, line 277, in call_callback
callback(*args, **kwargs)
File “C:\Users\avida\FreeWeibo\WordEmbeddingAnalysis.py”, line 202, in WordEmbed
sim_i_j = model.n_similarity(final_weibos[i], final_weibos[j])
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\core\frame.py”, line 3805, in getitem
indexer = self.columns.get_loc(key)
File “C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\pandas\core\indexes\base.py”, line 3805, in get_loc
raise KeyError(key) from err
The relevant part of the error is here: line 202. Check the size of your final_weibos dataframe.
temp_list = []
num_weibos = 10
sim_matrix = [0]*num_weibos
#print(sim_matrix)
if final_weibos.shape[0] < 10:
st.write(f'There are not enough final_weibos: {final_weibos.shape[0]}')
else:
for i in range(num_weibos):
for j in range(num_weibos):
sim_i_j = model.n_similarity(final_weibos[i], final_weibos[j])
temp_list.append(sim_i_j)
sim_matrix[i] = list(temp_list)
temp_list = []
#print(len(sim_matrix))
print(sim_matrix)
print(max(sim_matrix))
So this is the code where I introduce the Final_weibos.
import pandas as pd
import re
#print(final_weibos[0:3])
final_weibos_new = pd.read_csv("C:\\Users\\avida\\FreeWeibo\\final_weibos.csv", sep = '/t' ,header=None, engine = 'python')
#final_weibos_new.head()
final_weibos = final_weibos_new
final_weibos_new = final_weibos_new.values.tolist()
#x = final_weibos_new[1]
#newlist = [word for line in x for word in line.split(",")]
#print(newlist)
new_list =[]
for i in range(len(final_weibos_new)):
weibo_post = final_weibos_new[i]
weibo_post_words = [word for line in weibo_post for word in line.split(",")]
new_list.append(weibo_post_words)
##print(final_weibos[2])
##print(new_list[2])
final_weibos_new = new_list
print(final_weibos_new[2])
##print(len(final_weibos))
print(len(final_weibos_new))
What did the check say about its size? If it is smaller than 10 and you expect otherwise, check the csv file, check how the csv file is being read, etc.
What do you mean what did the check say?
The code I posted above was your code, plus a check to verify the size.
Yeah I looked at the csv and there is more than 10
this is the code for bringing the .csv
data = final_weibos
# opening the csv file in 'w+' mode
file = open('final_weibos.csv', 'w+', encoding='utf-8', newline ="")
# writing the data into the file
with file:
write = csv.writer(file)
write.writerows(data)
Did you follow the trail?
CSV File has > 10 lines
final_weibos_new has more than 10 rows
final_weibos has more than 10 rows
You need to look at what is happening with each line and verify the value of your data. Especially with dataframes, you may have some process changing a variable you didn’t expect since the variable is just a pointer.
For example, if one is not careful, there can be unintended edits to other variable somewhere. Here we set df2 = df1 then make a change to df1. This changes df2 as well.
import streamlit as st
import pandas as pd
df1 = pd.DataFrame({'name':['Alice','Bob'],'age':[27,32]})
st.write('df1')
st.write(df1)
df2 = df1
st.write('df2')
st.write(df2)
df1['name']=['Charlotte','Daren']
st.write('df2 after changing df1')
st.write(df2)
So when you encounter an error, first check the value of everything at that exact point right before the error. When you find something that isn’t what it should be, follow it through your code step-by-step to find out when/where it ceased to have the value you wanted.
What would be the reason I am getting the error.
Blockquote
UnboundLocalError: local variable ‘st’ referenced before assignment
Traceback:
File "C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 561, in _run_script
self._session_state.on_script_will_rerun(rerun_data.widget_states)File "C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\safe_session_state.py", line 72, in on_script_will_rerun
self._state.on_script_will_rerun(latest_widget_states)File "C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py", line 543, in on_script_will_rerun
self._call_callbacks()File "C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py", line 556, in _call_callbacks
self._new_widget_state.call_callback(wid)File "C:\Users\avida\anaconda3\envs\FreeWeibo_working\lib\site-packages\streamlit\runtime\state\session_state.py", line 277, in call_callback
callback(*args, **kwargs)File "C:\Users\avida\FreeWeibo\WordEmbeddingAnalysis.py", line 108, in WordEmbed
weibo_input = st.number_input(label, 0, max_num_weibos-1, value=0, step=1, key='
for the line of code we used in the past.
weibo_input = st.number_input(label, 0, max_num_weibos-1, value=0, step=1, key='weibo_input')
It means that line of code does not know what st is. You need to make sure Streamlit is imported. You may need to import it at the top, but within your function since you are importing functions to other scripts.
Wow I should have known that!
Just so you know its definitely the code
weibo_input = st.number_input(label, 0, max_num_weibos-1, value=0, step=1, key='weibo_input')
that is causing the callback error
Try getting the value of max_num_weibos outside of the input with a check to make sure it isn’t zero. Given what I saw when I ran the code, you should have escape routes out of the process to let you know when you have empty dataframes along the way.