4

I am trying to install the ernie and tokenizers on Windows, but for both cases it is showing the following error:

ERROR: Command errored out with exit status 1:
   command: 'c:\users\my pc\appdata\local\programs\python\python39\python.exe' 'c:\users\my pc\appdata\local\programs\python\python39\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' build_wheel 'C:\Users\MYPC~1\AppData\Local\Temp\tmpyfqm86h3'
       cwd: C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7
  Complete output (38 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib
  creating build\lib\tokenizers
  copying tokenizers\__init__.py -> build\lib\tokenizers
  running build_ext
  running build_rust
      Updating crates.io index
  cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --target x86_64-pc-windows-msvc --release --verbose -- --crate-type cdylib
  warning: unused manifest key: target.x86_64-apple-darwin.rustflags
     Compiling proc-macro2 v1.0.30
     Compiling unicode-xid v0.2.2
     Compiling syn v1.0.80
     Compiling autocfg v1.0.1
       Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.30\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=0b72d9facdf9b740 -C extra-filename=-0b72d9facdf9b740 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\proc-macro2-0b72d9facdf9b740" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow`
       Running `rustc --crate-name unicode_xid "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-xid-0.2.2\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=c5404e573edb1c5f -C extra-filename=-c5404e573edb1c5f --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow`
       Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.80\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b3126876d28730d4 -C extra-filename=-b3126876d28730d4 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\syn-b3126876d28730d4" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow`
       Running `rustc --crate-name autocfg "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\autocfg-1.0.1\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=077a4bc4a690ef7e -C extra-filename=-077a4bc4a690ef7e --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow`
     Compiling memchr v2.4.1
       Running `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.4.1\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=420338dd5ea363a5 -C extra-filename=-420338dd5ea363a5 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\memchr-420338dd5ea363a5" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow`
  error: linker `link.exe` not found
    |
    = note: The system cannot find the file specified. (os error 2)

  note: the msvc targets depend on the msvc linker but `link.exe` was not found

  note: please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option

  error: could not compile `syn` due to previous error

  Caused by:
    process didn't exit successfully: `rustc --crate-name build_script_build --edition=2018 "C:\Users\my pc\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.80\build.rs" --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b3126876d28730d4 -C extra-filename=-b3126876d28730d4 --out-dir "C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\build\syn-b3126876d28730d4" -L "dependency=C:\Users\my pc\AppData\Local\Temp\pip-install-504a8mvv\tokenizers_a45d458877334719a3bf67c944dce6d7\target\release\deps" --cap-lints allow` (exit code: 1)
  warning: build failed, waiting for other jobs to finish...
  error: build failed
  error: cargo failed with code: 101

  ----------------------------------------
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
wovano
  • 3,264
  • 4
  • 18
  • 43
Jaswanth Ch
  • 49
  • 1
  • 1
  • 3
  • 1
    Yous should copy and paste all lines of the error. Also, please include the OS information. – umitu Oct 17 '21 at 00:52
  • I fixed the code formatting for you, but this is not the full error message. Please copy-paste the actual error message you get into the question, at least the part from "Building wheel for tokenizers" until the end! This part only contains the "summary" which is that the command failed, however, it does not include the relevant information about what failed. This makes it difficult to answer the question. – wovano Oct 18 '21 at 18:22
  • Now that you've included the relevant information, the cause of the error is pretty clear, right? **_error: linker `link.exe` not found_** And the solution is also mentioned in the error message: ***please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option*** – wovano Oct 19 '21 at 11:03
  • Sorry the solution was not clear, could you please elaborate. I am using the visual studio code and i installed an extension called visual c++ option in visual studio code but what is vs 2013 and how do i ensure it. – Jaswanth Ch Oct 20 '21 at 16:14
  • You need Visual Studio (the compiler) version 2013 or higher to build this Python library. Visual Studio Code is an IDE. Although the name is similar it's something completely different! You can download Visual Studio [here](https://visualstudio.microsoft.com/visual-cpp-build-tools/). I recently did it to install another Python library. [This answer](https://stackoverflow.com/a/69565927/10669875) might be helpful as well. – wovano Oct 20 '21 at 16:20

5 Answers5

3

It need to install rust compiler in your environment, you can use the command:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

and then you can do it successfully.

AlanFok
  • 31
  • 2
1

I solved this problem by installing Visual Studio 2019 Community Edition with Python development workloads. I am not sure whether the Python development workloads were essential. Rust compiler has already been installed. (rustup-init.exe from www.rust-lang.org)

Edit: When I uninstalled Python 3.10, Visual Studio, and Rust completely and tried again with only Python 3.10 and Visual Studio Community edition with Python development workloads, it didn't work. But it succeeded after I installed Desktop Development with C++ workloads.

user67275
  • 1,748
  • 4
  • 31
  • 51
0

So what worked for me is twofold: -1. Install Rust -2. Install Visual Studio (latest version, 2022) and make sure to install BOTH Python workloads and Desktop Development C++ Workloads.

saladin1991
  • 103
  • 8
-1

It seems that the corresponding package isn't installed on your machine, you need to install wheel.
try these & let me know if it works...

  • pip3 install --upgrade pip
  • pip3 install --upgrade pip setuptools wheel

or pip3 install wheel.

Ali Abdi
  • 126
  • 4
  • 14
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Oct 18 '21 at 07:31
  • Could you explain why you think this is the problem/solution? – wovano Oct 18 '21 at 08:23
  • Sorry my question was clear, I edited the question. I think the problem is while building the wheel for tokenizers, for other packages the wheel is working absolutely fine – Jaswanth Ch Oct 18 '21 at 17:10
  • I had a problem like this before & I thought that `wheel` wasn't installed on your machine; for tokenizers, have you seen [this link](https://github.com/huggingface/transformers/issues/2831)? I think the solutions here will work. – Ali Abdi Oct 21 '21 at 08:22
-2

If you have not installed Java, install it and try again

nr spider
  • 101
  • 1
  • 8