-
Notifications
You must be signed in to change notification settings - Fork 11
How to build venus gui v2
There are various targets / build types:
- Building to run on a GX device itself, and displaying on the local display. i.e. Ekrano GX and/or Cerbo with GX Touch 50 or GX Touch 70.
- Building the WebAssembly, which is hosted on the GX device or the VRM server, but in the end runs in a browser and gets data over MQTT.
- Building for desktop, for development/testing purposes, it can get data over MQTT connection to a broker running on a GX device on the local LAN, or it can use a data simulator or mock mode.
This section contains information on how to build gui-v2 as an ARM executable to run on a GX device.
It gets easily outdated, you’ll have to replace references to Venus OS beta versions to the latest available one.
Requirements:
- Your host (= laptop) must have the Venus OS SDK installed. Note that that SDK is only available for hosts running linux, either bare metal or as VM. And requires an intel processor. So for example using a Apple M2 for building it is not going to work. To install that SDK, see the Venus OS SDK instructions. Make sure to use the most recent SDK. This also ensures that the Qt version used for building gui-v2 is the same as the version we use, and the same as on the GX.
- SSH access onto GX device.
- A GX device with recent firmware version (e.g. v3.50~13), instructions here.
The following instructions show how to install the Venus OS SDK to /opt/venus on your host (laptop):
# create venus dir for sdk:
$ cd /opt
$ sudo mkdir venus
# extract the sdk
$ cd ~ # or wherever you downloaded the sdk .sh
$ ./venus-dunfell-x86_64-arm-cortexa7hf-neon-vfpv4-toolchain-qte-v3.20~29.sh
Once you have installed the SDK, you can use it to build gui-v2 for GX devices:
# source the sdk env and build
$ . /opt/venus/dunfell-arm-cortexa7hf-neon-vfpv4/environment-setup-cortexa7hf-neon-vfpv4-ve-linux-gnueabi
$ cd ~/gui-v2 # or wherever your code repository is
~/gui-v2$ git checkout -b main origin/main
~/gui-v2$ git submodule update --init
~/gui-v2$ mkdir build
~/gui-v2$ cd build
~/gui-v2/build$ cmake -DCMAKE_BUILD_TYPE=MinSizeRel ../
~/gui-v2/build$ cmake --build . --config MinSizeRel
The above instructions should have built venus-gui-v2 executable under ~/gui-v2/build/bin directory. You can scp that executable to the device and run it directly.
~/gui-v2/build$ scp bin/venus-gui-v2 root@cerbogx:~/
~/gui-v2/build$ ssh root@cerbogx
root@einstein:~# ./venus-gui-v2
Normally, venus-gui-v2 is installed to /opt/victronenergy/gui-v2/ on device.
Note: you may wish to stop the existing gui (version 1) application:
svc -d /service/start-gui
or restart it:
svc -u /service/start-gui
This section contains information on how to build gui-v2 as a WASM executable to run within a web browser. There are four options:
- Building for WebAssembly on Linux host
- Building for WebAssembly on Windows host
- Building for WebAssembly on Mac OS X host
- Building for WebAssembly via GitHub Actions (CI)
Finally, this chapter contains two sections explaining how to install it on the GX device as well as on VRM:
- Patch and pack the WebAssembly files
- Installing the WebAssembly artifacts to GX device
Now and then we switch to newer versions of those. And then updating this wiki page is easily forgotten. Hence, before doing all the installing and building of the toolchain, first check what versions we use at the moment by checking the versions used in our wasm ci workflow.
This section contains information about how to build gui-v2 using a similar environment as is used to build official releases.
The canonical build steps can be found in the GitHub Action script: build-wasm.yml.
Installation instructions are using aqt for installing Qt. Of course it is also possible to install all using the normal Maintenance tool, here aqt is used because it is more suitable for automated installs.
Starting with a recent Ubuntu linux distribution:
sudo apt-get install -y g++ build-essential mesa-common-dev libssl-dev \
wget lsb libgl1-mesa-dev libxkbcommon-x11-0 libpulse-dev p7zip-full \
ninja-build dos2unix
As mentioned aqt is needed in order to follow the rest of the instructions:
pip install aqtinstall
The command yq is needed for running the later mentioned qt-fixes.sh script:
sudo snap install yq
Notice that the versions and install directory are set using environment variables. Make sure to source these first before copy/pasting the rest of the instructions:
export QT_VERSION=6.6.1
export EMSCRIPTEN=3.1.37
export OUTPUTDIR=/opt/hostedtoolcache
aqt install-qt linux desktop ${QT_VERSION} gcc_64 -m qtwebsockets qt5compat qtshadertools --outputdir ${OUTPUTDIR}/Qt
This install is needed for building QtMQTT later on.
aqt install-qt linux desktop ${QT_VERSION} wasm_singlethread -m qtwebsockets qt5compat qtshadertools --outputdir ${OUTPUTDIR}/Qt
aqt install-tool linux desktop tools_cmake --outputdir ${OUTPUTDIR}/Qt
If you've used the Maintenance tool for installation so far, this might not be needed. The aqt install however does miss some needed settings. It lacks some executable bits and has some paths wrong. On the Github Action side, the qt-fixes.sh shell script takes
care of this. If you have used the same OUTPUTDIR (/opt/hostedtoolcache) and you have yq installed, you can run the script: ./.github/patches/qt-fixes.sh to do these patches for you. Otherwise please check the script and do these patches manually.
cd ${OUTPUTDIR}
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install ${EMSCRIPTEN}
./emsdk activate ${EMSCRIPTEN}
cd ${OUTPUTDIR}
source "${OUTPUTDIR}/emsdk/emsdk_env.sh"
git clone https://github.com/qt/qtmqtt.git
cd qtmqtt
git checkout $QT_VERSION
mkdir build-qtmqtt && cd build-qtmqtt
export PATH=${OUTPUTDIR}/Qt/Tools/CMake/bin:$PATH
export QTDIR=${OUTPUTDIR}/Qt/$QT_VERSION/wasm_singlethread
${QTDIR}/bin/qt-configure-module ..
cmake --build .
cmake --install . --prefix ${QTDIR} --verbose
Finally, we are ready to actually build gui-v2 as a WASM executable (note that ${QTDIR} should have been exporting previously, see above):
git clone https://github.com/victronenergy/gui-v2.git
cd gui-v2
git submodule update --init
mkdir build-wasm
cd build-wasm
${QTDIR}/bin/qt-cmake -DCMAKE_BUILD_TYPE=MinSizeRel ..
cmake --build .
The build artifacts then need to be processed as per the "Patch and pack" instructions below before they can be installed to GX device.
First, install emscripten of the appropriate version (for Qt 6.6.1, emsdk install 3.1.37; emsdk activate 3.1.37).
Second, install Qt For WebAssembly 6.6.1 via the Qt Maintenance Tool.
Then, build and install the QtMqtt module manually (as a binary build isn't offered in the maintenance tool yet):
git clone https://github.com/qt/qtmqtt.git
cd qtmqtt
git checkout 6.6.1
mkdir build && cd build
CALL "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64
C:\Development\emscripten\emsdk\emsdk_env.bat
set PATH=C:\Development\Qt\Tools\CMake_64\bin;%PATH%
C:\Development\Qt\6.6.1\wasm_singlethread\bin\qt-configure-module.bat ..
C:\Development\Qt\Tools\CMake_64\bin\cmake.exe --build .
C:\Development\Qt\Tools\CMake_64\bin\cmake.exe --install . --verbose
Note that the vcvarsall.bat path may differ depending on which version of VS you have installed, e.g.:
CALL "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64
Note:
- Building qtmqtt 6.6.1 for wasm_singlethread on Windows 11 seems to have a couple of quirks.
- First, need to edit
~/Qt/6.6.1/wasm_singlethread/lib/cmake/Qt6BuildInternals/QtBuildInternalsExtra.cmaketo ensure that the Qt paths are properly escaped (e.g.C:\\Development\\Qt\\6.6.1\\wasm_singlethreadinstead ofC:\Development\Qt\6.6.1\wasm_singlethread - Second, need to specify an install prefix manually (e.g.
C:\Development\Qt\Tools\CMake_64\bin\cmake.exe --install . --prefix "C:\\Development\\Qt\\6.6.1\\wasm_singlethread" --verbosefor the final step). - See https://github.com/victronenergy/gui-v2/issues/441#issuecomment-1681609453 for full instructions.
Once you have installed and built QtMqtt for Qt For WebAssembly, you can build gui-v2 as per normal using QtCreator.
The build artifacts then need to be processed as per the "Patch and pack" instructions below before they can be installed to GX device.
The WebAssembly application needs to be served by a webserver.
The CerboGX has its own nginx HTTP server, and by default serves the content from /var/www/venus/gui-beta to the path http://<ip_address>/gui-beta/, which is why we install WebAssembly artifacts to GX device.
However, on desktop, you can serve it with emrun or another local webserver:
C:\Development\emscripten\emsdk\python\3.9.2-1_64bit\python.exe \
C:/Development/emscripten/emsdk/upstream/emscripten/emrun.py \
--browser chrome --port 30000 --no_emrun_detect
--serve_after_close C:/build-wasm/artifacts/wasm/index.html
Note: any simple web server should suffice, if you don't have emrun.
Also note that the WebAssembly build will try to connect to the localhost by default. When serving the build from your host machine rather than the GX device, you will need to tell it to connect to a GX device on your network, by adding ?mqtt=ws://gx.device.ip.address:9001 to the url.
First, install emscripten of the appropriate version (for Qt 6.6.1, emsdk install 3.1.37; emsdk activate 3.1.37).
Second, install Qt For WebAssembly 6.6.1 via the Qt Maintenance Tool.
Then, build and install the QtMqtt module manually (as a binary build isn't offered in the maintenance tool yet):
NOTE: when building for WebAssembly, run source /path/to/emsdk_env.sh before building to ensure that the emscripten toolchain is found, and use the qt-cmake and --prefix for Qt For WebAssembly (e.g. ~/Qt6/6.6.1/wasm_singlethread) instead of the desktop Qt.
# assumes Qt For WebAssembly 6.6.1 is installed under ~/Qt6/6.6.1/wasm_singlethread
cd ~/Qt6/
git clone https://github.com/qt/qtmqtt.git
cd qtmqtt
git checkout 6.6.1
cd ~/Qt6/
mkdir -p build-qtmqtt
# assumes you have installed emscripten to ~/emscripten/emsdk
source ~/emcripten/emsdk/emsdk_env.sh
cd ~/Qt6/build-qtmqtt
~/Qt6/6.6.1/wasm_singlethread/bin/qt-cmake ~/Qt6/qtmqtt
cmake --build .
cmake --install . --prefix ~/Qt6/6.6.1/wasm_singlethread --verbose
After building and installing the QtMqtt module as above, you can now build gui-v2 for WebAssembly, either using QtCreator, or via the following steps:
$ git clone https://github.com/victronenergy/gui-v2.git
$ cd gui-v2
$ ~/gui-v2$ git submodule update --init
$ ~/gui-v2$ mkdir build
$ ~/gui-v2$ cd build/
$ ~/gui-v2/build$ ~/Qt6/6.6.1/wasm_singlethread/bin/qt-cmake -DCMAKE_BUILD_TYPE=MinSizeRel ../
$ ~/gui-v2/build$ /opt/Qt/Tools/CMake/bin/cmake --build .
The build artifacts then need to be processed as per the "Patch and pack" instructions below before they can be installed to GX device.
If you do not wish to set up a WebAssembly build environment on your own system, you may instead use GitHub Actions to build gui-v2.
To do this, trigger the GitHub action for gui-v2 (this will happen automatically for all branches and tags in the canonical gui-v2 repository) and once it completes it will become visible under the Actions list. You can click on the completed action to see its details, including its Artifacts (which should include a zip file which contains pre-patched and packed artifacts, ready for installation to the GX device).
To repeat: if building gui-v2 for WebAssembly via GitHub Actions, there is no need to perform the "Patch and pack" instructions below; you should instead proceed directly to the installation steps.
Unless you use the CI system / GitHub Actions to build for WebAssembly, then regardless of which host you used to build the WebAssembly artifacts the important files need to be preprocessed before they are ready to be scp'd to the device.
The following script can be used for this purpose:
#!/bin/sh
# run this after cd build-wasm directory.
# artifacts will be under ~/gui-v2/build-wasm/artifacts/wasm
# ready for scp to device.
mkdir -p artifacts/wasm
cp venus-gui-v2.html venus-gui-v2.js venus-gui-v2.wasm qtloader.js ../images/victronenergy.svg ../.github/patches/index.html.patch artifacts/wasm/
mv artifacts/wasm/venus-gui-v2.html artifacts/wasm/index.html
dos2unix artifacts/wasm/index.html artifacts/wasm/index.html.patch
patch artifacts/wasm/index.html < artifacts/wasm/index.html.patch
grep -q -E '^var createQtAppInstance' artifacts/wasm/venus-gui-v2.js
sed -i "s%^var \(createQtAppInstance\)%window.\1%" artifacts/wasm/venus-gui-v2.js
cp ../.github/patches/Makefile artifacts/wasm/
cp ../LICENSE.txt artifacts/wasm/
cd artifacts/wasm
Once the WebAssembly build artifacts are ready, they can be installed to the device:
cd ~/gui-v2/build-wasm/artifacts/wasm
make
scp LICENSE.txt index.html qtloader.js venus-gui-v2.js venus-gui-v2.wasm.gz venus-gui-v2.wasm.sha256 victronenergy.svg root@cerbogx:/var/www/venus/gui-beta/
Note: you might need to make the filesystem writable first, via running # /opt/victronenergy/swupdate-scripts/resize2fs.sh on device.
Now reboot the GX device, and wait 10 minutes. VRM should be notified of the change to the gui-v2 version, so that it can be launched from VRM.
You may now launch gui-v2 either from your Local LAN (by browsing to https://gx.device.ip.address/gui-beta/) or from VRM (by clicking the gui-v2 button when logged into the VRM website and viewing the relevant installation).
This section contains information on how to build gui-v2 as a desktop executable on either Windows, Mac OS X, or Linux.
This is mostly for development/testing purposes, as desktop builds are not a supported target. It can be useful for developers as desktop builds still have access to the --mock backend, or unsecured MQTT broker backend via --mqtt gx.device.ip.address argument.
When building Venus OS, a specific version of Qt is used (currently Qt 6.6.2, but this can change without this wiki being updated). Before attempting the instructions below, check if the mentioned versions are outdated (either by: logging into a GX device using SSH and checking ls /usr/lib | grep Qt6; OR by checking this file: https://github.com/victronenergy/venus/blob/master/configs/dunfell/repos.conf#L5).
Note that Dunfell is also a version (in this case, of OpenEmbedded), and in the future that will updated, so instructions which include a path with Dunfell in it may also be outdated in future.
First, install Qt 6.6.2 including Qt Shader Tools and also (from Additional Libraries) Qt WebSockets, using the Qt Maintenance Tool. Second, build and install QtMqtt manually:
git clone https://github.com/qt/qtmqtt.git
cd qtmqtt
git checkout 6.6.2
mkdir build && cd build
CALL "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64
C:\Development\Qt\6.6.2\msvc2019_64\bin\qt-configure-module.bat ..
C:\Development\Qt\Tools\CMake_64\bin\cmake.exe --build .
C:\Development\Qt\Tools\CMake_64\bin\cmake.exe --install . --verbose
Note: for debug builds, add --config Debug to the last two cmake commands.
Finally, you can build gui-v2 using QtCreator.
You can run gui-v2 with the --mock argument to test with the mock backend.
(This is what the Venus OS team members usually use: mac and linux.)
First, install Qt 6.6.2 including Qt Shader Tools and also (from Additional Libraries) Qt WebSockets, using the Qt Maintenance Tool. Second, build and install QtMqtt manually:
cd ~/Qt6/
git clone https://github.com/qt/qtmqtt.git
cd qtmqtt
git checkout 6.6.2
cd ~/Qt6/
mkdir -p build-qtmqtt
cd ~/Qt6/build-qtmqtt
~/Qt6/6.6.2/macos/bin/qt-cmake ~/Qt6/qtmqtt # or g++ instead of macos
cmake --build .
cmake --install . --prefix ~/Qt6/6.6.2/macos --verbose # or g++ instead of macos
Finally, you can build gui-v2 using QtCreator, or via the following steps:
$ git clone https://github.com/victronenergy/gui-v2.git
$ cd gui-v2
$ ~/gui-v2$ git submodule update --init
$ ~/gui-v2$ mkdir build
$ ~/gui-v2$ cd build/
$ ~/gui-v2/build$ ~/Qt6/6.6.2/macos/bin/qt-cmake -DCMAKE_BUILD_TYPE=MinSizeRel ../
$ ~/gui-v2/build$ cmake --build .
You can run gui-v2 with the --mock argument to test with the mock backend. (On Mac OS X, this is via: open bin/venus-gui-v2.app --args --mock.)
On Mac OS X, you may need to manually set the Qt6_DIR environment variable via export Qt6_DIR=/Users/username/Qt/6.6.3/macos/lib/cmake/Qt6/ if the above instructions do not work (typically: if you are using cmake from ~/Qt/Tools/CMake/bin rather than via the version-specific qt-cmake wrapper).
For troubleshooting, read these issues of others that went through this before you: