diff --git a/docs/source/using-executorch-building-from-source.md b/docs/source/using-executorch-building-from-source.md index 973e9c5f55b..59f3365f661 100644 --- a/docs/source/using-executorch-building-from-source.md +++ b/docs/source/using-executorch-building-from-source.md @@ -7,91 +7,72 @@ like Make, Ninja or Xcode. For information, see [cmake-generators(7)](https://cm ## System Requirements ### Operating System -We've tested these instructions on the following systems, although they should -also work in similar environments. - - -Linux (x86_64) -- CentOS 8+ -- Ubuntu 20.04.6 LTS+ -- RHEL 8+ - -macOS (x86_64/ARM64) -- Big Sur (11.0)+ - -Windows (x86_64) -- Windows Subsystem for Linux (WSL) with any of the Linux options - -### Software +ExecuTorch is tested on the following systems, although it should also work in similar environments. + + * Linux (x86_64) + * CentOS 8+ + * Ubuntu 20.04.6 LTS+ + * RHEL 8+ + * macOS (x86_64/ARM64) + * Big Sur (11.0)+ + * Windows (x86_64) + * Windows Subsystem for Linux (WSL) with any of the Linux options + * Windows 10+ with Visual Studio 2022+ (experimental) + +### Software Requirements * `conda` or another virtual environment manager - - We recommend `conda` as it provides cross-language + - `conda` is recommended as it provides cross-language support and integrates smoothly with `pip` (Python's built-in package manager) - Otherwise, Python's built-in virtual environment manager `python venv` is a good alternative. * `g++` version 7 or higher, `clang++` version 5 or higher, or another C++17-compatible toolchain. * `python` version 3.10-3.12 +* `Xcode Command Line Tools` (macOS only) * `ccache` (optional) - A compiler cache that speeds up recompilation +Additional dependencies will be installed automatically when running the [Python installation](#building-the-python-package). Note that the cross-compilable core runtime code supports a wider range of toolchains, down to C++17. See the [Runtime Overview](runtime-overview.md) for portability details. ## Environment Setup - -### Clone ExecuTorch - + Clone the ExecuTorch repository from GitHub and create a conda environment as follows. Venv can be used in place on conda. ```bash - # Clone the ExecuTorch repo from GitHub - git clone -b viable/strict https://github.com/pytorch/executorch.git && cd executorch + git clone -b viable/strict https://github.com/pytorch/executorch.git + cd executorch + conda create -yn executorch python=3.10.0 + conda activate executorch ``` -### Create a Virtual Environment - -Create and activate a Python virtual environment: - ```bash - python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip - ``` +
-Or alternatively, [install conda on your machine](https://conda.io/projects/conda/en/latest/user-guide/install/index.html). Then, create a Conda environment named "executorch". - ```bash - conda create -yn executorch python=3.10.0 && conda activate executorch - ``` +## Building the Python package + To build and install the ExecuTorch Python components, used for PTE creation and Python runtime bindings, run the following command. + This will install the ExecuTorch python package and its dependencies into the active Python environment. -## Install ExecuTorch pip package from source ```bash - # Install ExecuTorch pip package and its dependencies, as well as - # development tools like CMake, and backend support for XNNPACK and CoreML. - # If developing on a Mac, make sure to install the Xcode Command Line Tools first. - # Intel-based macOS systems require building PyTorch from source (see below) + # Install ExecuTorch pip package and its dependencies. ./install_executorch.sh ``` - See the [PyTorch instructions](https://github.com/pytorch/pytorch#installation) on how to build PyTorch from source. + The `install_executorch.sh` script supports the following flags: - Use the [`--use-pt-pinned-commit` flag](../../install_executorch.py) to install ExecuTorch with an existing PyTorch build: + * `--clean`: Removes build artifacts. + * `--editable`: Install the ExecuTorch python package in editable mode (see [Editable Install](#editable-install)). + * `--minimal`: Install only the minimal set of dependencies required to run ExecuTorch. Do not install dependencies for examples. + * `--use-pt-pinned-commit`: Install the pinned PyTorch commit. When not specified, the latest PyTorch nightly build is installed. - ```bash - ./install_executorch.sh --use-pt-pinned-commit - ``` - - For Intel-based macOS systems, use the [`--use-pt-pinned-commit --minimal` flags](../../install_executorch.py): - ```bash - ./install_executorch.sh --use-pt-pinned-commit --minimal - ``` + For Intel-based macOS systems, use `--use-pt-pinned-commit --minimal`. As PyTorch does not provide pre-built binaries for Intel Mac, installation requires building PyTorch from source. Instructions can be found in [PyTorch Installation](https://github.com/pytorch/pytorch#installation). - Notice that only XNNPACK and CoreML backends are supported by default. You can enable additional backends or disable default backends by setting the corresponding CMake flags: + Note that only the XNNPACK and CoreML backends are built by default. Additional backends can be enabled or disabled by setting the corresponding CMake flags: ```bash # Enable the MPS backend CMAKE_ARGS="-DEXECUTORCH_BUILD_MPS=ON" ./install_executorch.sh ``` - ```bash - # Disable the XNNPACK backend - CMAKE_ARGS="-DEXECUTORCH_BUILD_XNNPACK=OFF" ./install_executorch.sh - ``` - - For development mode, run the command with `--editable`, which allows us to modify Python source code and see changes reflected immediately. + ### Editable Install + For development, include the `--editable` flag, which allows for local changes to ExecuTorch Python code to be reflected without a re-install. Note that when C++ files are modified, you will need to re-run the full installation to reflect the changes. ```bash ./install_executorch.sh --editable @@ -100,10 +81,8 @@ Or alternatively, [install conda on your machine](https://conda.io/projects/cond pip install -e . --no-build-isolation ``` - If C++ files are being modified, you will still have to reinstall ExecuTorch from source. - > **_WARNING:_** -> Some modules can't be imported directly in editable mode. This is a known [issue](https://github.com/pytorch/executorch/issues/9558) and we are actively working on a fix for this. To workaround this: +> Some modules can't be imported directly in editable mode. This is a known [issue](https://github.com/pytorch/executorch/issues/9558) and we are actively working on a fix for this. To work around this: > ```bash > # This will fail > python -c "from executorch.exir import CaptureConfig" @@ -129,31 +108,15 @@ Or alternatively, [install conda on your machine](https://conda.io/projects/cond > > The `--clean` command removes build artifacts, pip outputs, and also clears the ccache if it's installed, ensuring a completely fresh build environment. -## Build ExecuTorch C++ runtime from source +
-ExecuTorch's CMake build system covers the pieces of the runtime that are -likely to be useful to embedded systems users. - -- `libexecutorch.a`: The core of the ExecuTorch runtime. Does not contain any - operator/kernel definitions or backend definitions. -- `libportable_kernels.a`: The implementations of ATen-compatible operators, - following the signatures in `//kernels/portable/functions.yaml`. -- `libportable_kernels_bindings.a`: Generated code that registers the contents - of `libportable_kernels.a` with the runtime. - - NOTE: This must be linked into your application with a flag like - `-Wl,-force_load` or `-Wl,--whole-archive`. It contains load-time functions - that automatically register the kernels, but linkers will often prune those - functions by default because there are no direct calls to them. -- `executor_runner`: An example tool that runs a `.pte` program file using all - `1` values as inputs, and prints the outputs to stdout. It is linked with - `libportable_kernels.a`, so the program may use any of the operators it - implements. +## Building the C++ Runtime +The ExecuTorch C++ runtime is built using CMake. It can be compiled standalone to run examples, added as a CMake dependency, or cross-compiled for Android, iOS, or embedded platforms. -### Configure the CMake build +### Configuring -Follow these steps after cloning or pulling the upstream repo, since the build -dependencies may have changed. +Configuration should be done after cloning, pulling the upstream repo, or changing build options. Once this is done, you won't need to do it again until you pull from the upstream repo or modify any CMake-related files. ```bash # cd to the root of the executorch repo @@ -165,24 +128,79 @@ cd executorch (mkdir cmake-out && cd cmake-out && cmake ..) ``` -Once this is done, you don't need to do it again until you pull from the upstream repo again, or if you modify any CMake-related files. +### Building -### CMake build options +Build all targets with `cmake --build`. -The release build offers optimizations intended to improve performance and reduce binary size. It disables program verification and executorch logging, and adds optimizations flags. ```bash --DCMAKE_BUILD_TYPE=Release +# cd to the root of the executorch repo +cd executorch + +# Build using the configuration that you previously generated under the +# `cmake-out` directory. +# +# NOTE: The `-j` argument specifies how many jobs/processes to use when +# building, and tends to speed up the build significantly. It's typical to use +# "core count + 1" as the `-j` value. +cmake --build cmake-out -j9 ``` -To further optimize the release build for size, use both: +> **_TIP:_** For faster rebuilds, consider installing ccache (see [Compiler Cache section](#compiler-cache-ccache) above). On first builds, ccache populates its cache. Subsequent builds with the same compiler flags can be significantly faster. + +### Build Presets + +ExecuTorch provides fine-grained control over what is built, as described in [Build Options](#build-options). These options are grouped into CMake presets to cover common scenarios, while providing the ability to override individual options. Presets can be specified when configuring CMake by specifying `--preset [name]` when configuring. + +Preset values for common scenarios are listed below. Using a platform preset is recommended to avoid needing to specify many fine-grained build options. + + * `arm-baremetal` - Build for bare-metal ARM targets. + * `ios` - Build features and backends common for iOS targets. + * `macos` - Build features and backends common for Mac targets. + * `linux` - Build features and backends for Linux targets. + * `llm` - Build Large Language Model-specific features. + * `profiling` - Build the ExecuTorch runtime with profiling enabled. + * `zephyr` - Build for Zephyr RTOS. + ```bash --DCMAKE_BUILD_TYPE=Release \ --DEXECUTORCH_OPTIMIZE_SIZE=ON +# Configure the build with the ios preset. +cmake .. --preset ios +``` + +### CMake Targets and Libraries + +To link against the ExecuTorch framework from CMake, the following top-level targets are exposed: + + * `executorch::backends`: Contains all configured backends. + * `executorch::extensions`: Contains all configured extensions. + * `executorch::kernels`: Contains all configured kernel libraries. + +The backends, extensions, and kernels included in these targets are controlled by the various `EXECUTORCH_` CMake options specified by the build. Using these targets will automatically pull in the required dependencies to use the configured features. + +### Running an Example Model + +The example `executor_runner` binary can be used to run a model and sanity-check the build. Run the following commands to generate and run a simple model. +You should see the message "Model executed successfully" followed by the output values. + +``` bash +python -m examples.portable.scripts.export --model_name="add" +./cmake-out/executor_runner --model_path add.pte +``` + +``` +I 00:00:00.000526 executorch:executor_runner.cpp:82] Model file add.pte is loaded. +I 00:00:00.000595 executorch:executor_runner.cpp:91] Using method forward +I 00:00:00.000612 executorch:executor_runner.cpp:138] Setting up planned buffer 0, size 48. +I 00:00:00.000669 executorch:executor_runner.cpp:161] Method loaded. +I 00:00:00.000685 executorch:executor_runner.cpp:171] Inputs prepared. +I 00:00:00.000764 executorch:executor_runner.cpp:180] Model executed successfully. +I 00:00:00.000770 executorch:executor_runner.cpp:184] 1 outputs: +Output 0: tensor(sizes=[1], [2.]) ``` -#### Compiler Cache (ccache) -ExecuTorch automatically detects and enables [ccache](https://ccache.dev/) if it's installed on your system. This significantly speeds up recompilation by caching previously compiled objects: +### Compiler Cache (ccache) + +ExecuTorch automatically detects and enables [ccache](https://ccache.dev/) if it's installed. This significantly speeds up recompilation by caching previously compiled objects: - If ccache is detected, you'll see: `ccache found and enabled for faster builds` - If ccache is not installed, you'll see: `ccache not found, builds will not be cached` @@ -205,177 +223,223 @@ No additional configuration is needed - the build system will automatically use See [CMakeLists.txt](https://github.com/pytorch/executorch/blob/main/CMakeLists.txt) -### Build the runtime components +
-Build all targets with +## Build Options -```bash -# cd to the root of the executorch repo -cd executorch +CMake options can be used to for fine-grained control of build type, control which features are built, and configure functionality, such as logging. Options are typically specified during CMake configuration. Default values of each option are set by the active preset, but can be overridden by specifying the option when configuring. -# Build using the configuration that you previously generated under the -# `cmake-out` directory. -# -# NOTE: The `-j` argument specifies how many jobs/processes to use when -# building, and tends to speed up the build significantly. It's typical to use -# "core count + 1" as the `-j` value. -cmake --build cmake-out -j9 +Note that many build options require other options to be enabled. This may require enabling multiple options to enable a given feature. The CMake build output will provide an error message when a required option is not enabled. + +#### Build Type + +The CMake build is typically set to `Debug` or `Release`. For production use or profiling, release mode should be used to improve performance and reduce binary size. It disables program verification and executorch logging and adds optimizations flags. The `EXECUTORCH_OPTIMIZE_SIZE` flag can be used to further optimize for size with a small performance tradeoff. + +```bash +# Specify build type during CMake configuration +cmake .. -DCMAKE_BUILD_TYPE=Release ``` -> **_TIP:_** For faster rebuilds, consider installing ccache (see [Compiler Cache section](#compiler-cache-ccache) above). On first builds, ccache populates its cache. Subsequent builds with the same compiler flags can be significantly faster. +#### Backends -## Use an example binary `executor_runner` to execute a .pte file +Typically, each hardware backend exposes a CMake option to control whether the backend is built. See backend-specific documentation for more details. -First, generate a .pte file, either by exporting an example model or following -the instructions in [Model Export and Lowering](using-executorch-export.md). + * `EXECUTORCH_BUILD_CADENCE` - Build the Cadence DSP backend. + * `EXECUTORCH_BUILD_COREML` - Build the Apple CoreML backend. + * `EXECUTORCH_BUILD_CORTEX_M` - Build the ARM Cortex-M backend. + * `EXECUTORCH_BUILD_MPS` - Build the Apple Metal Performance Shader backend. + * `EXECUTORCH_BUILD_NEURON` - Build the MediaTek Neuron backend. + * `EXECUTORCH_BUILD_OPENVINO` - Build the Intel OpenVINO backend. + * `EXECUTORCH_BUILD_QNN` - Build the Qualcomm AI Engine backend. + * `EXECUTORCH_BUILD_VGF` - Build the ARM VGF backend. + * `EXECUTORCH_BUILD_VULKAN` - Build the Vulkan GPU backend. + * `EXECUTORCH_BUILD_XNNPACK` - Build the XNNPACK CPU backend. -To generate a simple model file, run the following command from the ExecuTorch directory. It -will create a file named "add.pte" in the current directory. -``` -python -m examples.portable.scripts.export --model_name="add" -``` -Then, pass it to the command line tool: ```bash -./cmake-out/executor_runner --model_path add.pte +# Build the XNNPACK and Vulkan backends. +cmake .. -DEXECUTORCH_BUILD_XNNPACK=ON -DEXECUTORCH_BUILD_VULKAN=ON ``` -You should see the message "Model executed successfully" followed -by the output values. +#### Extensions -``` -I 00:00:00.000526 executorch:executor_runner.cpp:82] Model file add.pte is loaded. -I 00:00:00.000595 executorch:executor_runner.cpp:91] Using method forward -I 00:00:00.000612 executorch:executor_runner.cpp:138] Setting up planned buffer 0, size 48. -I 00:00:00.000669 executorch:executor_runner.cpp:161] Method loaded. -I 00:00:00.000685 executorch:executor_runner.cpp:171] Inputs prepared. -I 00:00:00.000764 executorch:executor_runner.cpp:180] Model executed successfully. -I 00:00:00.000770 executorch:executor_runner.cpp:184] 1 outputs: -Output 0: tensor(sizes=[1], [2.]) -``` +ExecuTorch extensions provide optional functionality outside of the core runtime. As the core runtime is designed to run in constrained environments, these features are typically disabled by default. Extensions include higher-level APIs (Module and Tensor), multi-threading support (Threadpool), training, and more. -### CMake Targets + * `EXECUTORCH_BUILD_EXTENSION_APPLE` - Build the Apple extension. This provides Swift and Objective-C bindings, log routing, and platform integration with Mac and iOS. See [Using ExecuTorch on iOS](using-executorch-ios.md). + * `EXECUTORCH_BUILD_EXTENSION_DATA_LOADER` - Build the data loader extension. Provides classes to load PTEs from files or buffers. + * `EXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR` - Build the flat tensor extension. Provides functionality to load and save tensor data in .ptd format. + * `EXECUTORCH_BUILD_EXTENSION_LLM` - Build the Large Language Model extension. Provides LLM-specific functionality, such as tokenizer APIs. See [Working with LLMs](llm/getting-started.md). + * `EXECUTORCH_BUILD_EXTENSION_LLM_APPLE` - Build the Large Language Model Apple extensions. + * `EXECUTORCH_BUILD_EXTENSION_LLM_RUNNER` - Build the Large Language Model runner extension. + * `EXECUTORCH_BUILD_EXTENSION_MODULE` - Build the Module API extension. See [High-Level APIs](using-executorch-cpp.md#high-level-apis). + * `EXECUTORCH_BUILD_EXTENSION_TENSOR` - Build the Tensor API extension. Provides convenience APIs for creating and managing tensors. See [High-Level APIs](using-executorch-cpp.md#high-level-apis) and [extension/tensor](https://github.com/pytorch/executorch/tree/main/extension/tensor). + * `EXECUTORCH_BUILD_EXTENSION_TRAINING` - Build the training extension. This is experimental. + * `EXECUTORCH_BUILD_EXTENSION_EVALUE_UTIL` - Build the EValue utility extension. Provides a method to print EValue objects. See [print_evalue.h](https://github.com/pytorch/executorch/blob/main/extension/evalue_util/print_evalue.h). + * `EXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL` - Build the runner utility extension. Provides utility methods for running models, such as allocating input and output tensor memory and generating inputs. See [executor_runner.cpp](https://github.com/pytorch/executorch/blob/main/examples/portable/executor_runner/executor_runner.cpp) for example usage. -To link against the ExecuTorch framework from CMake, the following top-level targets are exposed: + ``` +# Enable the data loader extension. +cmake .. -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON + ``` - * `executorch::backends`: Contains all configured backends. - * `executorch::extensions`: Contains all configured extensions. - * `executorch::kernels`: Contains all configured kernel libraries. +#### Logging -The backends, extensions, and kernels included in these targets are controlled by the various `EXECUTORCH_` CMake options specified by the build. +Logging is enabled by default in debug builds and disabled in release. When enabled, the default log level is Info. Both log enable and level can be overriden with options. See [Logging](using-executorch-runtime-integration.md#logging). Disabling logging and decreasing log verbosity will reduce binary size by stripping unused strings from the build. -## Build ExecuTorch for Windows +* `EXECUTORCH_ENABLE_LOGGING` - Enable or disable framework log messages. +* `EXECUTORCH_LOG_LEVEL` - The minimum log level to emit. One of `debug`, `info`, `error`, or `fatal`. -This document outlines the current known working build instructions for building and validating ExecuTorch on a Windows machine. + ``` +# Enable logging at debug +cmake .. -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_LOG_LEVEL=debug + ``` -This demo uses the -[MobileNet v2](https://pytorch.org/vision/main/models/mobilenetv2.html) model to classify images using the [XNNPACK](https://github.com/google/XNNPACK) backend. +#### Output Libraries -Note that all commands should be executed on Windows powershell in administrator mode. +To link against the runtime from outside of the CMake ecosystem, the runtime can be first built with CMake and then linked directly. A few of the relevant top-level targets are described below. Note that this is a more involved process than using CMake and is only recommended when using CMake is not viable. -### Pre-requisites +- `libexecutorch.a`: The core of the ExecuTorch runtime. Does not contain any + operator/kernel definitions or backend definitions. +- `libportable_kernels.a`: The implementations of ATen-compatible operators, + following the signatures in `//kernels/portable/functions.yaml`. +- `libportable_kernels_bindings.a`: Generated code that registers the contents + of `libportable_kernels.a` with the runtime. + - NOTE: This must be linked into your application with a flag like + `-Wl,-force_load` or `-Wl,--whole-archive`. It contains load-time functions + that automatically register the kernels, but linkers will often prune those + functions by default because there are no direct calls to them. + `libportable_kernels.a`, so the program may use any of the operators it + implements. -#### 1. Install Miniconda for Windows -Install miniconda for Windows from the [official website](https://docs.conda.io/en/latest/miniconda.html). +Backends typically introduce additional targets. See backend-specific documentation for more details. -#### 2. Install Git for Windows -Install Git for Windows from the [official website](https://git-scm.com/download/win). +
-#### 3. Install ClangCL for Windows -Install ClangCL for Windows from the [official website](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170). +## Cross-Compiling for Android +### Pre-requisites +- Set up a Python environment and clone the ExecuTorch repository, as described in [Environment Setup](#environment-setup). +- Install the [Android SDK](https://developer.android.com/studio). Android Studio is recommended. +- Install the [Android NDK](https://developer.android.com/ndk). + - Option 1: Install via [Android Studio](https://developer.android.com/studio/projects/install-ndk). + - Option 2: Download from [NDK Downloads](https://developer.android.com/ndk/downloads). -### Create the Conda Environment -To check if conda is detected by the powershell prompt, try `conda list` or `conda --version` +### Building the AAR -If conda is not detected, you could run the powershell script for conda named `conda-hook.ps1`. -To verify that Conda is available in the in the powershell environment, run try `conda list` or `conda --version`. -If Conda is not available, run conda-hook.ps1 as follows: -```bash -$miniconda_dir\\shell\\condabin\\conda-hook.ps1 -``` -where `$miniconda_dir` is the directory where you installed miniconda -This is `“C:\Users\\AppData\Local”` by default. +With the NDK installed, the `build_android_library.sh` script will build the ExecuTorch Java AAR. This file contains the ExecuTorch Java bindings +and native code. See [Using the AAR File](using-executorch-android.md#using-aar-file) for usage. -#### Create and activate the conda environment: ```bash -conda create -yn et python=3.12 -conda activate et +export ANDROID_ABIS=arm64-v8a +export BUILD_AAR_DIR=aar-out +mkdir -p $BUILD_AAR_DIR +sh scripts/build_android_library.sh ``` -### Check Symlinks -Set the following environment variable to enable symlinks: -```bash -git config --global core.symlinks true -``` +### Building the Example Runner -### Set up ExecuTorch -Clone ExecuTorch from the [official GitHub repository](https://github.com/pytorch/executorch). +The native executor runner can be cross-compiled for android and deployed via ADB. This step is intended as +an example of CMake cross compilation and is not necessary for integration into an app. ```bash -git clone --recurse -submodules https://github.com/pytorch/executorch.git -``` +# Run the following lines from the `executorch/` folder +./install_executorch.sh --clean +mkdir cmake-android-out && cd cmake-android-out + +# point -DCMAKE_TOOLCHAIN_FILE to the location where ndk is installed +cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a .. -### Run the Setup Script +cd .. +cmake --build cmake-android-out -j9 -Currently, there are a lot of components that are not buildable on Windows. The below instructions install a very minimal ExecuTorch which can be used as a sanity check. +adb shell mkdir -p /data/local/tmp/executorch +# push the binary to an Android device +adb push cmake-android-out/executor_runner /data/local/tmp/executorch +# push the model file +adb push add.pte /data/local/tmp/executorch -#### Move into the `executorch` directory -```bash -cd executorch +adb shell "/data/local/tmp/executorch/executor_runner --model_path /data/local/tmp/executorch/add.pte" ``` -#### (Optional) Run a --clean script prior to running the .bat file. +
+ +## Cross-Compiling for iOS + +For iOS, we'll build [frameworks](https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle) instead of static libraries. The frameworks contain the compiled ExecuTorch runtime and public headers. + +### Pre-requisites + +* Install Xcode from the +[Mac App Store](https://apps.apple.com/app/xcode/id497799835) and install +the Command Line Tools using the terminal. + ```bash -./install_executorch.bat --clean +xcode-select --install ``` -#### Run the setup script. -You could run the .bat file or the python script. +### Building + +1. Build the frameworks: + ```bash -./install_executorch.bat -# OR -# python install_executorch.py +./scripts/build_apple_frameworks.sh ``` -### Export MobileNet V2 +Run the above command with `--help` flag to learn more on how to build additional backends +(like [Core ML](backends-coreml.md), [MPS](backends-mps.md) or XNNPACK), etc. +Note that some backends may require additional dependencies and certain versions of Xcode and iOS. +See backend-specific documentation for more details. -Create the following script named export_mv2.py +2. Copy over the generated `.xcframework` bundles to your Xcode project, link them against +your targets and don't forget to add an extra linker flag `-all_load`. -```bash -from torchvision.models import mobilenet_v2 -from torchvision.models.mobilenetv2 import MobileNet_V2_Weights +Check out the [iOS Demo App](https://github.com/pytorch-labs/executorch-examples/tree/main/mv3/apple/ExecuTorchDemo) tutorial for more info. -mv2 = mobilenet_v2(weights=MobileNet_V2_Weights.DEFAULT) # This is torch.nn.Module +
-import torch -from executorch.exir import to_edge -from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner +## Building on Windows -model = mv2.eval() # turn into evaluation mode +ExecuTorch provides experimental support for native Windows builds. -example_inputs = (torch.randn((1, 3, 224, 224)),) # Necessary for exporting the model +> **_NOTE:_** All commands should be executed on Windows powershell in administrator mode. -exported_graph = torch.export.export(model, example_inputs) # Core Aten graph +### Environment Setup -edge = to_edge(exported_graph) # Edge Dialect +#### Pre-requisites -edge_delegated = edge.to_backend(XnnpackPartitioner()) # Parts of the graph are delegated to XNNPACK +1. Install miniconda for Windows from the [official website](https://docs.conda.io/en/latest/miniconda.html). +2. Install Git for Windows from the [official website](https://git-scm.com/download/win). +3. Install ClangCL for Windows from the [official website](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170) or through a [Visual Studio](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170) or [Visual Studio Code](https://code.visualstudio.com/docs/cpp/config-clang-mac) installation. -executorch_program = edge_delegated.to_executorch() # ExecuTorch program +#### Clone and Configure Environment -pte_path = "mv2_xnnpack.pte" +```bash +git config --global core.symlinks true +git clone --recurse -submodules https://github.com/pytorch/executorch.git +cd executorch +conda create -yn et python=3.12 +conda activate et +``` -with open(pte_path, "wb") as file: - executorch_program.write_to_file(file) # Serializing into .pte file +If Conda is not available, run conda-hook.ps1, where `$miniconda_dir` is the directory where miniconda is installed. +This is `“C:\Users\\AppData\Local”` by default. + +```bash +$miniconda_dir\\shell\\condabin\\conda-hook.ps1 ``` -#### Run the export script to create a `mv2_xnnpack.pte` file. +### Build the Python Package + +Run `install_executorch.bat` to build and install the ExecuTorch Python package and runtime bindings. ```bash -python .\\export_mv2.py +cd executorch +./install_executorch.bat ``` -### Build and Install C++ Libraries + Binaries +> **_NOTE_** Many components are not currently buildable on Windows. These instructions install a very minimal ExecuTorch which can be used as a sanity check. + +### Build the C++ Runtime + ```bash del -Recurse -Force cmake-out; ` cmake . ` @@ -395,103 +459,45 @@ cmake . ` -Bcmake-out; ` cmake --build cmake-out -j64 --target install --config Release ``` -where `$miniconda_dir` is the directory where you installed miniconda -This is `“C:\Users\\AppData\Local”` by default. - -### Run Mobilenet V2 model with XNNPACK delegation - -```bash -.\\cmake-out\\backends\\xnnpack\\Release\\xnn_executor_runner.exe --model_path=.\\mv2_xnnpack.pte -``` - -The expected output would print a tensor of size 1x1000, containing values of class scores. - -```bash -Output 0: tensor(sizes=[1, 1000], [ - -0.50986, 0.30064, 0.0953904, 0.147726, 0.231205, 0.338555, 0.206892, -0.0575775, … ]) -``` - -Congratulations! You've successfully set up ExecuTorch on your Windows device and ran a MobileNet V2 model. -Now, you can explore and enjoy the power of ExecuTorch on your own Windows device! - -## Cross compilation -Following are instruction on how to perform cross compilation for Android and iOS. +> **_NOTE_** `$miniconda_dir` is the directory where you installed miniconda. This is `“C:\Users\\AppData\Local”` by default. -### Android +### Running an Example Model -#### Building executor_runner shell binary -- Prerequisite: [Android NDK](https://developer.android.com/ndk), choose one of the following: - - Option 1: Download Android Studio by following the instructions to [install ndk](https://developer.android.com/studio/projects/install-ndk). - - Option 2: Download Android NDK directly from [here](https://developer.android.com/ndk/downloads). +To validate the installation by running a model, create a file named export_mv2.py. Then, run the powershell commands to export and run the model. +The expected output is a tensor of size 1x1000, containing class scores. -Assuming Android NDK is available, run: -```bash -# Run the following lines from the `executorch/` folder -./install_executorch.sh --clean -mkdir cmake-android-out && cd cmake-android-out - -# point -DCMAKE_TOOLCHAIN_FILE to the location where ndk is installed -cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a .. - -cd .. -cmake --build cmake-android-out -j9 - -adb shell mkdir -p /data/local/tmp/executorch -# push the binary to an Android device -adb push cmake-android-out/executor_runner /data/local/tmp/executorch -# push the model file -adb push add.pte /data/local/tmp/executorch +```py +# export_mv2.py +import torch +from executorch.exir import to_edge_transform_and_lower +from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner +from torchvision.models import mobilenet_v2 +from torchvision.models.mobilenetv2 import MobileNet_V2_Weights -adb shell "/data/local/tmp/executorch/executor_runner --model_path /data/local/tmp/executorch/add.pte" -``` +mv2 = mobilenet_v2(weights=MobileNet_V2_Weights.DEFAULT).eval() +example_inputs = (torch.randn((1, 3, 224, 224)),) -#### Building AAR for app integration from source -- Prerequisite: Android NDK from the previous section, and Android SDK (Android Studio is recommended). +program = to_edge_transform_and_lower( + torch.export.export(model, example_inputs) +).to_executorch() -Assuming Android NDK and SDK is available, run: -```bash -export ANDROID_ABIS=arm64-v8a -export BUILD_AAR_DIR=aar-out -mkdir -p $BUILD_AAR_DIR -sh scripts/build_android_library.sh +with open("mv2_xnnpack.pte", "wb") as file: + executorch_program.write_to_file(file) ``` -This script will build the AAR, which contains the Java API and its corresponding JNI library. Please see -[this documentation](using-executorch-android.md#using-aar-file) for usage. - -### iOS - -For iOS we'll build [frameworks](https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle) instead of static libraries, that will also contain the public headers inside. - -1. Install Xcode from the -[Mac App Store](https://apps.apple.com/app/xcode/id497799835) and then install -the Command Line Tools using the terminal: - ```bash -xcode-select --install +python .\\export_mv2.py +.\\cmake-out\\backends\\xnnpack\\Release\\xnn_executor_runner.exe --model_path=.\\mv2_xnnpack.pte ``` -2. Build the frameworks: - ```bash -./scripts/build_apple_frameworks.sh +Output 0: tensor(sizes=[1, 1000], [ + -0.50986, 0.30064, 0.0953904, 0.147726, 0.231205, 0.338555, 0.206892, -0.0575775, … ]) ``` -Run the above command with `--help` flag to learn more on how to build additional backends -(like [Core ML](backends-coreml.md), [MPS](backends-mps.md) or XNNPACK), etc. -Note, some backends may require additional dependencies and certain versions of Xcode and iOS. - -3. Copy over the generated `.xcframework` bundles to your Xcode project, link them against -your targets and don't forget to add an extra linker flag `-all_load`. - -Check out the [iOS Demo App](https://github.com/pytorch-labs/executorch-examples/tree/main/mv3/apple/ExecuTorchDemo) tutorial for more info. - - -## Next steps - -You have successfully cross-compiled `executor_runner` binary to iOS and Android platforms. You can start exploring advanced features and capabilities. Here is a list of sections you might want to read next: +## Next Steps -* [Selective build](kernel-library-selective-build.md) to build the runtime that links to only kernels used by the program, which can provide significant binary size savings. +* [Selective Build](kernel-library-selective-build.md) to link only kernels used by the program. This can provide significant binary size savings. * Tutorials on building [Android](https://github.com/pytorch-labs/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app) and [iOS](https://github.com/pytorch-labs/executorch-examples/tree/main/mv3/apple/ExecuTorchDemo) demo apps. * Tutorials on deploying applications to embedded devices such as [ARM Cortex-M/Ethos-U](backends-arm-ethos-u.md) and [XTensa HiFi DSP](backends-cadence.md).