New trouble Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

This is on the cloud and I am struggling with how to fix my issue - new here so not sure how to do this work.

[01:55:41] πŸ“¦ Processing dependencies...
[01:55:41] πŸ“¦ Apt dependencies were installed from /mount/src/uncancelled/packages.txt using apt-get.
Hit:1 http://deb.debian.org/debian bullseye InRelease
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 https://packages.microsoft.com/debian/11/prod bullseye InRelease [3629 B]
Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [256 kB]
Get:6 https://packages.microsoft.com/debian/11/prod bullseye/main amd64 Packages [125 kB]
Get:7 https://packages.microsoft.com/debian/11/prod bullseye/main arm64 Packages [27.4 kB]
Fetched 505 kB in 0s (1032 kB/s)
[01:55:43] β›“ Spinning up manager process...
Reading package lists...[2023-10-29 01:55:43.776810] 
Building dependency tree...[2023-10-29 01:55:43.922506] 
Reading state information...[2023-10-29 01:55:43.922903] 
The following additional packages will be installed:
  cargo gdb libbabeltrace1 libboost-regex1.74.0 libc6-dbg libdebuginfod1
  libgit2-1.1 libhttp-parser2.9 libipt2 libllvm11 libmbedcrypto3 libmbedtls12
  libmbedx509-0 libpython3.9 libsource-highlight-common libsource-highlight4v5
  libstd-rust-1.48 libstd-rust-dev libz3-4 rust-gdb
Suggested packages:
  cargo-doc gdb-doc gdbserver rust-doc rust-src lld-11
Recommended packages:
  cargo
The following NEW packages will be installed:
  cargo gdb libbabeltrace1 libboost-regex1.74.0 libc6-dbg libdebuginfod1
  libgit2-1.1 libhttp-parser2.9 libipt2 libllvm11 libmbedcrypto3 libmbedtls12
  libmbedx509-0 libpython3.9 libsource-highlight-common libsource-highlight4v5
  libstd-rust-1.48 libstd-rust-dev libz3-4 rust-gdb rustc
0 upgraded, 21 newly installed, 0 to remove and 5 not upgraded.
Need to get 94.8 MB of archives.
After this operation, 409 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 libhttp-parser2.9 amd64 2.9.4-4+deb11u1 [22.2 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libmbedcrypto3 amd64 2.16.9-0.1 [219 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libmbedx509-0 amd64 2.16.9-0.1 [109 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 libmbedtls12 amd64 2.16.9-0.1 [139 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 libgit2-1.1 amd64 1.1.0+dfsg.1-4+deb11u1 [444 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 libz3-4 amd64 4.8.10-1 [6949 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 libllvm11 amd64 1:11.0.1-2 [17.9 MB]
Get:8 http://deb.debian.org/debian bullseye/main amd64 libstd-rust-1.48 amd64 1.48.0+dfsg1-2 [20.7 MB]
Get:9 http://deb.debian.org/debian bullseye/main amd64 libstd-rust-dev amd64 1.48.0+dfsg1-2 [29.5 MB]
Get:10 http://deb.debian.org/debian bullseye/main amd64 rustc amd64 1.48.0+dfsg1-2 [2049 kB]
Get:11 http://deb.debian.org/debian bullseye/main amd64 cargo amd64 0.47.0-3+b1 [2870 kB]
Get:12 http://deb.debian.org/debian bullseye/main amd64 libbabeltrace1 amd64 1.5.8-1+b3 [174 kB]
Get:13 http://deb.debian.org/debian bullseye/main amd64 libdebuginfod1 amd64 0.183-1 [27.4 kB]
Get:14 http://deb.debian.org/debian bullseye/main amd64 libipt2 amd64 2.0.3-1 [43.7 kB]
Get:15 http://deb.debian.org/debian bullseye/main amd64 libpython3.9 amd64 3.9.2-1 [1691 kB]
Get:16 http://deb.debian.org/debian bullseye/main amd64 libsource-highlight-common all 3.1.9-3 [79.7 kB]
Get:17 http://deb.debian.org/debian bullseye/main amd64 libboost-regex1.74.0 amd64 1.74.0-9 [516 kB]
Get:18 http://deb.debian.org/debian bullseye/main amd64 libsource-highlight4v5 amd64 3.1.9-3+b1 [259 kB]
Get:19 http://deb.debian.org/debian bullseye/main amd64 gdb amd64 10.1-1.7 [3395 kB]
Get:20 http://deb.debian.org/debian-security bullseye-security/main amd64 libc6-dbg amd64 2.31-13+deb11u7 [7531 kB]
Get:21 http://deb.debian.org/debian bullseye/main amd64 rust-gdb all 1.48.0+dfsg1-2 [165 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 94.8 MB in 1s (150 MB/s)
Selecting previously unselected package libhttp-parser2.9:amd64.
(Reading database ... 22031 files and directories currently installed.)
Preparing to unpack .../00-libhttp-parser2.9_2.9.4-4+deb11u1_amd64.deb ...
Unpacking libhttp-parser2.9:amd64 (2.9.4-4+deb11u1) ...
Selecting previously unselected package libmbedcrypto3:amd64.
Preparing to unpack .../01-libmbedcrypto3_2.16.9-0.1_amd64.deb ...
Unpacking libmbedcrypto3:amd64 (2.16.9-0.1) ...
Selecting previously unselected package libmbedx509-0:amd64.
Preparing to unpack .../02-libmbedx509-0_2.16.9-0.1_amd64.deb ...
Unpacking libmbedx509-0:amd64 (2.16.9-0.1) ...
Selecting previously unselected package libmbedtls12:amd64.
Preparing to unpack .../03-libmbedtls12_2.16.9-0.1_amd64.deb ...
Unpacking libmbedtls12:amd64 (2.16.9-0.1) ...
Selecting previously unselected package libgit2-1.1:amd64.
Preparing to unpack .../04-libgit2-1.1_1.1.0+dfsg.1-4+deb11u1_amd64.deb ...
Unpacking libgit2-1.1:amd64 (1.1.0+dfsg.1-4+deb11u1) ...
Selecting previously unselected package libz3-4:amd64.
Preparing to unpack .../05-libz3-4_4.8.10-1_amd64.deb ...
Unpacking libz3-4:amd64 (4.8.10-1) ...
Selecting previously unselected package libllvm11:amd64.
Preparing to unpack .../06-libllvm11_1%3a11.0.1-2_amd64.deb ...
Unpacking libllvm11:amd64 (1:11.0.1-2) ...
Selecting previously unselected package libstd-rust-1.48:amd64.
Preparing to unpack .../07-libstd-rust-1.48_1.48.0+dfsg1-2_amd64.deb ...
Unpacking libstd-rust-1.48:amd64 (1.48.0+dfsg1-2) ...
Selecting previously unselected package libstd-rust-dev:amd64.
Preparing to unpack .../08-libstd-rust-dev_1.48.0+dfsg1-2_amd64.deb ...
Unpacking libstd-rust-dev:amd64 (1.48.0+dfsg1-2) ...
Selecting previously unselected package rustc.
Preparing to unpack .../09-rustc_1.48.0+dfsg1-2_amd64.deb ...
Unpacking rustc (1.48.0+dfsg1-2) ...
Selecting previously unselected package cargo.
Preparing to unpack .../10-cargo_0.47.0-3+b1_amd64.deb ...
Unpacking cargo (0.47.0-3+b1) ...
Selecting previously unselected package libbabeltrace1:amd64.
Preparing to unpack .../11-libbabeltrace1_1.5.8-1+b3_amd64.deb ...
Unpacking libbabeltrace1:amd64 (1.5.8-1+b3) ...
Selecting previously unselected package libdebuginfod1:amd64.
Preparing to unpack .../12-libdebuginfod1_0.183-1_amd64.deb ...
Unpacking libdebuginfod1:amd64 (0.183-1) ...
Selecting previously unselected package libipt2.
Preparing to unpack .../13-libipt2_2.0.3-1_amd64.deb ...
Unpacking libipt2 (2.0.3-1) ...
Selecting previously unselected package libpython3.9:amd64.
Preparing to unpack .../14-libpython3.9_3.9.2-1_amd64.deb ...
Unpacking libpython3.9:amd64 (3.9.2-1) ...
Selecting previously unselected package libsource-highlight-common.
Preparing to unpack .../15-libsource-highlight-common_3.1.9-3_all.deb ...
Unpacking libsource-highlight-common (3.1.9-3) ...
Selecting previously unselected package libboost-regex1.74.0:amd64.
Preparing to unpack .../16-libboost-regex1.74.0_1.74.0-9_amd64.deb ...
Unpacking libboost-regex1.74.0:amd64 (1.74.0-9) ...
Selecting previously unselected package libsource-highlight4v5.
Preparing to unpack .../17-libsource-highlight4v5_3.1.9-3+b1_amd64.deb ...
Unpacking libsource-highlight4v5 (3.1.9-3+b1) ...
Selecting previously unselected package gdb.
Preparing to unpack .../18-gdb_10.1-1.7_amd64.deb ...
Unpacking gdb (10.1-1.7) ...
Selecting previously unselected package libc6-dbg:amd64.
Preparing to unpack .../19-libc6-dbg_2.31-13+deb11u7_amd64.deb ...
Unpacking libc6-dbg:amd64 (2.31-13+deb11u7) ...
Selecting previously unselected package rust-gdb.
Preparing to unpack .../20-rust-gdb_1.48.0+dfsg1-2_all.deb ...
Unpacking rust-gdb (1.48.0+dfsg1-2) ...
Setting up libdebuginfod1:amd64 (0.183-1) ...
Setting up libsource-highlight-common (3.1.9-3) ...
Setting up libc6-dbg:amd64 (2.31-13+deb11u7) ...
Setting up libz3-4:amd64 (4.8.10-1) ...
Setting up libpython3.9:amd64 (3.9.2-1) ...
Setting up libllvm11:amd64 (1:11.0.1-2) ...
Setting up libboost-regex1.74.0:amd64 (1.74.0-9) ...
Setting up libipt2 (2.0.3-1) ...
Setting up libstd-rust-1.48:amd64 (1.48.0+dfsg1-2) ...
Setting up libbabeltrace1:amd64 (1.5.8-1+b3) ...
Setting up libmbedcrypto3:amd64 (2.16.9-0.1) ...
Setting up libhttp-parser2.9:amd64 (2.9.4-4+deb11u1) ...
Setting up libsource-highlight4v5 (3.1.9-3+b1) ...
Setting up libmbedx509-0:amd64 (2.16.9-0.1) ...
Setting up libmbedtls12:amd64 (2.16.9-0.1) ...
Setting up libstd-rust-dev:amd64 (1.48.0+dfsg1-2) ...
Setting up gdb (10.1-1.7) ...
Setting up rustc (1.48.0+dfsg1-2) ...
Setting up libgit2-1.1:amd64 (1.1.0+dfsg.1-4+deb11u1) ...
Setting up rust-gdb (1.48.0+dfsg1-2) ...
Setting up cargo (0.47.0-3+b1) ...
Processing triggers for libc-bin (2.31-13+deb11u7) ...

──────────────────────────────────────── pip ───────────────────────────────────────────

Collecting transformers==3.1.0
  Downloading transformers-3.1.0-py3-none-any.whl (884 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 884.0/884.0 KB 32.9 MB/s eta 0:00:00[2023-10-29 01:55:55.922813] 
Collecting bs4
  Downloading bs4-0.0.1.tar.gz (1.1 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting google.generativeai
  Downloading google_generativeai-0.2.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 KB 207.3 MB/s eta 0:00:00[2023-10-29 01:55:56.354241] 
Collecting pandas
  Downloading pandas-2.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.3/12.3 MB 209.0 MB/s eta 0:00:00[2023-10-29 01:55:56.827029] 
Collecting numpy
  Downloading numpy-1.26.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 195.7 MB/s eta 0:00:00[2023-10-29 01:55:57.430118] 
Collecting sentencepiece!=0.1.92
  Downloading sentencepiece-0.1.99-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 204.1 MB/s eta 0:00:00[2023-10-29 01:55:57.692291] 
Collecting tokenizers==0.8.1.rc2
  Downloading tokenizers-0.8.1rc2.tar.gz (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.4/97.4 KB 111.6 MB/s eta 0:00:00[2023-10-29 01:55:58.153740] 
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting requests
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 KB 196.5 MB/s eta 0:00:00[2023-10-29 01:56:02.888309] 
Collecting regex!=2019.12.17
  Downloading regex-2023.10.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (773 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 773.3/773.3 KB 249.5 MB/s eta 0:00:00[2023-10-29 01:56:03.897815] 
Collecting sacremoses
  Downloading sacremoses-0.0.53.tar.gz (880 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 880.6/880.6 KB 260.8 MB/s eta 0:00:00[2023-10-29 01:56:03.943642] 
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting filelock
  Downloading filelock-3.13.0-py3-none-any.whl (11 kB)
Collecting tqdm>=4.27
  Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 KB 136.9 MB/s eta 0:00:00[2023-10-29 01:56:04.375425] 
Collecting packaging
  Downloading packaging-23.2-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 KB 192.9 MB/s eta 0:00:00[2023-10-29 01:56:04.431076] 
Collecting beautifulsoup4
  Downloading beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.0/143.0 KB 142.1 MB/s eta 0:00:00[2023-10-29 01:56:04.482079] 
Collecting google-api-core
  Downloading google_api_core-2.12.0-py3-none-any.whl (121 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 KB 151.0 MB/s eta 0:00:00[2023-10-29 01:56:04.648195] 
Collecting google-ai-generativelanguage==0.3.3
  Downloading google_ai_generativelanguage-0.3.3-py3-none-any.whl (267 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 267.9/267.9 KB 144.0 MB/s eta 0:00:00[2023-10-29 01:56:04.774293] 
Collecting protobuf
  Downloading protobuf-4.24.4-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.6/311.6 KB 243.6 MB/s eta 0:00:00[2023-10-29 01:56:05.291132] 
Collecting google-auth
  Downloading google_auth-2.23.3-py2.py3-none-any.whl (182 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 182.3/182.3 KB 148.6 MB/s eta 0:00:00[2023-10-29 01:56:05.439609] 
Collecting proto-plus<2.0.0dev,>=1.22.0
  Downloading proto_plus-1.22.3-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.1/48.1 KB 176.9 MB/s eta 0:00:00[2023-10-29 01:56:05.552582] 
Collecting pytz>=2020.1
  Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.5/502.5 KB 265.0 MB/s eta 0:00:00[2023-10-29 01:56:05.808024] 
Collecting python-dateutil>=2.8.2
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 234.9 MB/s eta 0:00:00[2023-10-29 01:56:05.864363] 
Collecting tzdata>=2022.1
  Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 KB 216.4 MB/s eta 0:00:00[2023-10-29 01:56:05.915652] 
Collecting six>=1.5
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting soupsieve>1.2
  Downloading soupsieve-2.5-py3-none-any.whl (36 kB)
Collecting googleapis-common-protos<2.0.dev0,>=1.56.2
  Downloading googleapis_common_protos-1.61.0-py2.py3-none-any.whl (230 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 230.9/230.9 KB 143.5 MB/s eta 0:00:00[2023-10-29 01:56:06.206189] 
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting cachetools<6.0,>=2.0.0
  Downloading cachetools-5.3.2-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 KB 158.0 MB/s eta 0:00:00[2023-10-29 01:56:06.387476] 
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.0.7-py3-none-any.whl (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 KB 223.0 MB/s eta 0:00:00[2023-10-29 01:56:06.502736] 
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.5/139.5 KB 122.0 MB/s eta 0:00:00[2023-10-29 01:56:06.687829] 
Collecting idna<4,>=2.5
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 KB 169.2 MB/s eta 0:00:00[2023-10-29 01:56:06.745319] 
Collecting certifi>=2017.4.17
  Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 KB 236.6 MB/s eta 0:00:00[2023-10-29 01:56:06.796382] 
Collecting click
  Downloading click-8.1.7-py3-none-any.whl (97 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 KB 144.8 MB/s eta 0:00:00[2023-10-29 01:56:06.871731] 
Collecting joblib
  Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.2/302.2 KB 139.7 MB/s eta 0:00:00[2023-10-29 01:56:06.964516] 
Collecting grpcio<2.0dev,>=1.33.2
  Downloading grpcio-1.59.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 255.7 MB/s eta 0:00:00[2023-10-29 01:56:08.288577] 
Collecting grpcio-status<2.0.dev0,>=1.33.2
  Downloading grpcio_status-1.59.0-py3-none-any.whl (14 kB)
Collecting pyasn1<0.6.0,>=0.4.6
  Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 KB 222.2 MB/s eta 0:00:00[2023-10-29 01:56:08.515902] 
Building wheels for collected packages: tokenizers, bs4, sacremoses
  Building wheel for tokenizers (pyproject.toml): started
  Building wheel for tokenizers (pyproject.toml): still running...
  Building wheel for tokenizers (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  Γ— Building wheel for tokenizers (pyproject.toml) did not run successfully.
  β”‚ exit code: 1
  ╰─> [80 lines of output]
      /tmp/pip-build-env-8rc0e27c/overlay/lib/python3.9/site-packages/setuptools/dist.py:314: InformationOnly: Normalizing '0.8.1.rc2' to '0.8.1rc2'
        self.metadata.version = self._normalize_version(
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/tokenizers
      copying tokenizers/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers
      creating build/lib.linux-x86_64-cpython-39/tokenizers/models
      copying tokenizers/models/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/models
      creating build/lib.linux-x86_64-cpython-39/tokenizers/decoders
      copying tokenizers/decoders/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/decoders
      creating build/lib.linux-x86_64-cpython-39/tokenizers/normalizers
      copying tokenizers/normalizers/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/normalizers
      creating build/lib.linux-x86_64-cpython-39/tokenizers/pre_tokenizers
      copying tokenizers/pre_tokenizers/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/pre_tokenizers
      creating build/lib.linux-x86_64-cpython-39/tokenizers/processors
      copying tokenizers/processors/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/processors
      creating build/lib.linux-x86_64-cpython-39/tokenizers/trainers
      copying tokenizers/trainers/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/trainers
      creating build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/__init__.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/char_level_bpe.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/byte_level_bpe.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/bert_wordpiece.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/base_tokenizer.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/implementations/sentencepiece_bpe.py -> build/lib.linux-x86_64-cpython-39/tokenizers/implementations
      copying tokenizers/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers
      copying tokenizers/models/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/models
      copying tokenizers/decoders/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/decoders
      copying tokenizers/normalizers/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/normalizers
      copying tokenizers/pre_tokenizers/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/pre_tokenizers
      copying tokenizers/processors/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/processors
      copying tokenizers/trainers/__init__.pyi -> build/lib.linux-x86_64-cpython-39/tokenizers/trainers
      running build_ext
      running build_rust
          Updating crates.io index
          Updating git repository `https://github.com/n1t0/rayon-cond`
       Downloading crates ...
        Downloaded proc-macro-hack v0.5.20+deprecated
        Downloaded quote v1.0.33
        Downloaded ansi_term v0.12.1
        Downloaded cfg-if v0.1.10
        Downloaded version_check v0.9.4
        Downloaded strsim v0.8.0
        Downloaded atty v0.2.14
        Downloaded typetag-impl v0.1.8
        Downloaded indoc-impl v0.3.6
        Downloaded number_prefix v0.3.0
        Downloaded itoa v1.0.9
        Downloaded inventory-impl v0.1.11
        Downloaded memoffset v0.9.0
        Downloaded pyo3cls v0.9.2
        Downloaded scopeguard v1.2.0
        Downloaded vec_map v0.8.2
        Downloaded paste-impl v0.1.18
        Downloaded crossbeam-deque v0.8.3
        Downloaded typetag v0.1.8
        Downloaded ppv-lite86 v0.2.17
        Downloaded unindent v0.1.11
        Downloaded aho-corasick v0.7.20
        Downloaded cfg-if v1.0.0
        Downloaded inventory v0.1.11
        Downloaded indoc v0.3.6
        Downloaded lazy_static v1.4.0
        Downloaded indicatif v0.14.0
        Downloaded autocfg v1.1.0
        Downloaded paste v0.1.18
        Downloaded ghost v0.1.14
      error: failed to parse manifest at `/home/adminuser/.cargo/registry/src/github.com-1ecc6299db9ec823/ghost-0.1.14/Cargo.toml`
      
      Caused by:
        failed to parse the `edition` key
      
      Caused by:
        this version of Cargo is older than the `2021` edition, and only supports `2015` and `2018` editions.
      error: `cargo metadata --manifest-path Cargo.toml --format-version 1` failed with code 101
      -- Output captured from stdout:
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
  Building wheel for bs4 (setup.py): started
  Building wheel for bs4 (setup.py): finished with status 'done'
  Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1256 sha256=805c8bc4141b0ef1ead8b3e129a365ae4e20c7bddb1d3d366441c6d126dd0c4e
  Stored in directory: /tmp/pip-ephem-wheel-cache-cwmgi0wb/wheels/73/2b/cb/099980278a0c9a3e57ff1a89875ec07bfa0b6fcbebb9a8cad3
  Building wheel for sacremoses (setup.py): started
  Building wheel for sacremoses (setup.py): finished with status 'done'
  Created wheel for sacremoses: filename=sacremoses-0.0.53-py3-none-any.whl size=895241 sha256=c1e2d57b4ebe3de9d118515eb810e3ec3af1b7158e2675e95088709f2a41c425
  Stored in directory: /tmp/pip-ephem-wheel-cache-cwmgi0wb/wheels/12/1c/3d/46cf06718d63a32ff798a89594b61e7f345ab6b36d909ce033
Successfully built bs4 sacremoses
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
WARNING: You are using pip version 22.0.3; however, version 23.3.1 is available.
You should consider upgrading via the '/home/adminuser/venv/bin/python -m pip install --upgrade pip' command.
Checking if Streamlit is installed

Do you really need such an old version of tokenizers?

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.