AppTest and Mocking API Without a secrets.toml File Present

Using Streamlit
2

Hey @digitalghost-dev thanks for the separate post and detailed explanation with links. I took a look through the app code and the hosted app and spent a little time playing around with it. This is cool!! Nice app :slight_smile: Here are a few thoughts and findings that might help you unblock:

Finding issues with tests

I found that printing the AppTest contents is often a good way to figure out what’s going on with the test output when something unexpected is happening. I can slightly modify your first test to do this (note I added some dummy secret values too):

def test_title_area():
	at = AppTest.from_file("streamlit_app.py", default_timeout=1000)
	at.secrets["mapbox"] = {"mapbox_key": "<api key>"}
	at.secrets["gcp_service_account"] = {"client_email": "test@test.com", "token_uri": "some-token-uri"}
	at.run()
	print(at.main)
	assert "Premier League Statistics / 2023-24" in at.title[0].value
	assert "Current Round: " in at.subheader[0].value

Now I run:

pytest -v --disable-warnings -k test_title_area

I can see that the app prints an exception and since I did -v I get the exception printed underneath it too. With this particular test, I see that Google doesn’t like my mocked service account info: google.auth.exceptions.MalformedError: The private_key field was not found in the service account info.

More on that a bit later.

Modular testing

Since you already worked on decomposing your app into components and have several fairly distinct tabs, it might make sense to test those independently. I built a simple test for the StadiumMapSection class.

Notably, I also found that plotly will build a full mapbox chart object even without a real mapbox token, so you can do some running and inspection with a dummy token just fine. Here’s the test I built, which uses AppTest.from_function() to only run the stadium component code and passes:

def test_stadium_map():
	def draw_stadium_map():
		import pandas as pd
		from components.stadiums_map_section import StadiumMapSection
		stms = StadiumMapSection()
		stadium_df = pd.DataFrame({
			"latitude": [51.88],
			"longitude": [-0.42],
			"stadium": ["Kenilworth Road"],
			"team": ["Luton Town"],
		})
		stms.display(stadium_df)
	
	at = AppTest.from_function(draw_stadium_map, default_timeout=1000)
	at.secrets["mapbox"] = {"mapbox_key": "<test api key>"}
	at.run()
	# No exception printed
	assert not at.exception
	# Correct subheader printed
	assert at.subheader[0].value == "Location of Stadiums"

	# Parse out and inspect the chart data (this is still too hacky until we have native support)
	import json
	chart_data = json.loads(at.main.children[1].proto.figure.spec)
	assert chart_data["data"][0]["lat"] == [51.88]
	assert chart_data["data"][0]["hovertext"] == ['Kenilworth Road']

If you liked this approach, I think you could do something similar for the other components.

Easier testing for the whole app - de-composing the monolith query function

As I pointed out above, I ran into trouble mocking your whole app run because of the way the main app file currently tries to load the GCP credentials, then reuses those credentials several places. In particular, I noticed you have the bigquery_connection() function that has 9 return values, mostly dataframes used in the rest of your app.

With a bit of refactoring, I think you could decompose this and make it much easier to build tests for the rest of your app. For example, you could move all the connection building and query logic to its own file with more broken out functions for each dataframe like so:

# components/connections.py

import firebase_admin  # type: ignore
import pandas as pd
import streamlit as st
from firebase_admin import firestore  # type: ignore
from google.cloud import bigquery
from google.oauth2 import service_account  # type: ignore

@st.cache_resource
def firestore_connection():
	credentials = service_account.Credentials.from_service_account_info(
        st.secrets["gcp_service_account"]
    )
	if not firebase_admin._apps:
		firebase_admin.initialize_app()

	return firestore.Client(credentials=credentials)

@st.cache_data(ttl=600)
def run_query(query):
    credentials = service_account.Credentials.from_service_account_info(
        st.secrets["gcp_service_account"]
    )
    query_job = bigquery.Client(credentials=credentials).query(query)
    raw_data = query_job.result()
    data = [dict(data) for data in raw_data]
    return data

@st.cache_resource
def get_standings():
	standings_data = run_query(
		"""
            SELECT rank, logo, team, points, wins, draws, loses, goals_for, goals_against, goal_difference
            FROM `premier_league_dataset.standings`
			ORDER BY rank ASC;
        """
	)
	return pd.DataFrame(data=standings_data)

# ... similar for the other queries

(BTW, I see you followed the tutorial for bigquery connection in Streamlit - I am hoping we can update it soon to use st.connection which might make this a bit easier)

Once you do this update, all the credential handling logic is abstracted out of the main app file and it’s much easier to mock just the database call and expected return value and test your app code against that. Here’s a simple example that assumes the file above exists ^, for the real use case you probably want to run tests of your various tabs and the transforms you do on the dataframes.

# Use patch to mock the return value of get_standings() from above
@patch("components.connections.get_standings")
def test_standings(get_standings):
	def draw_standings():
        # This function is just for demo purpose, for a real test you should build AppTest on some of your app logic :)
		import streamlit as st
		from components.connections import get_standings
		df = get_standings()
		st.dataframe(df)

	# Now you can set a mock return value like this
	get_standings.return_value = pd.DataFrame({
		"team": ["Tottenham", "Manchester"],
		"standings": [5, 2],
	})
	at = AppTest.from_function(draw_standings).run()
    # And test that it gets used by the function
	assert at.dataframe[0].value.iloc[0][0] == "Tottenham"

Alternate approaches / e2e testing

Alternatively, if you want to test that the database and everything is working e2e in your CI, it may make sense to inject the actual GCP credentials as a secret in GitHub Actions and actually run the queries and test against real data in the CI. The limitation in this mocking approach is that it wont catch any mismatches between your actual database schema / ingested data and the app logic.

Setting that up is outside the scope of help I can provide since I’m not an expert, but the starter docs from GitHub are here. If you want to get really crazy, it looks like google supports some keyless approach too (i can’t tell if this totally makes sense for this use case or not).

Hope all this is helpful and makes sense! It was fun and useful for me to dig into this example a bit and understand how to best apply AppTest. Fair warning I may not be able to dig in deep for further follow up questions but if there’s any quick clarification I can provide, I’ll try to do so. Maybe others can jump in to help too. Thanks and good luck!!

1 Like