4041 Commits

Author SHA1 Message Date
Simon Glass
eb45f1ce17 binman: Update cbfstool
Update to a newer version of this tool, 4.22.01. This runs OK with the
current binman tests and matches the one in CI.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2024-07-26 08:01:06 -06:00
Brandon Maier
f0a166e096 tools: patman: fix pip install with Python 3.12
Installing patman with `cd ./tools/patman && pip install -e .` fails
with the error below.

As described in the error output below, the license line is not allowed
to be only defined in the setup.py. We remove the 'license' field
entirely, as the Python Packaging User Guide recommends using projects
classifiers instead[1] and we already set the GPL-2.0+ classifier.

> $ cd ./tools/patman && pip install -e .
> Obtaining file:///.../u-boot/tools/patman
>   Installing build dependencies ... done
>   Checking if build backend supports build_editable ... done
>   Getting requirements to build editable ... error
>   error: subprocess-exited-with-error
>
>   × Getting requirements to build editable did not run successfully.
>   │ exit code: 1
>   ╰─> [61 lines of output]
>       /tmp/pip-build-env-mqjvnmz8/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:76:
>           _MissingDynamic: `license` defined outside of `pyproject.toml` is ignored.
>       !!
>
>       ********************************************************************************
>       The following seems to be defined outside of `pyproject.toml`:
>
>       `license = 'GPL-2.0+'`
>
>       According to the spec (see the link below), however, setuptools CANNOT
>       consider this value unless `license` is listed as `dynamic`.
>
>       https://packaging.python.org/en/latest/specifications/pyproject-toml/#declaring-project-metadata-the-project-table
>
>       To prevent this problem, you can list `license` under `dynamic` or alternatively
>       remove the `[project]` table from your file and rely entirely on other means of
>       configuration.
>       ********************************************************************************
>
>       !!

[1] https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license

Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
c57d406134 qconfig: Move operation check into parse_args()
Put the check for an operation being provided into the parse_args()
function, to reduce the size of main().

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
e23a583940 qconfig: Drop col argument from Slots()
This is not needed since the progress indicator has the object. Use that
instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
948d0b45f5 qconfig: Use the Color object in Progress
Since the Progress class has the required object, use it from there
instead of passing it around.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
ab1bfd4b8f qconfig: Move the last two operations into their own functions
Put the summary and database-writing code into separate functions to
reduce the size of main().

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
0229eeef7b qconfig: Move progress output into the class
Rather than create these outputs separately, put them in the class so
that the main program doesn't need to deal with them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
6c2a4385ff qconfig: Move commit code into a separate function
Reduce the size of main() by putting this code into its own function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
eb1df3fde3 qconfig: Move all move_config code into move_config()
Move the setup and completion code into the move_config() function so
it is all in one place.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
630a9c9a47 qconfig: Add a return value to do_find_config()
Return an exit code so we can use this function like do_tests().
Refactor the caller to handle this.

Reduce the size of main() by putting this code into its own function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
cc628f582a qconfig: Move imply into a separate function
Reduce the size of main() by putting this code into its own function,
with the usage message staying in main().

Tidy up the comments for do_imply_config() while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
d63357eefc qconfig: Add a return value to do_scan_source()
Return an exit code so we can use this function like do_tests().
Refactor the caller to handle this.

Reduce the size of main() by putting this code into its own function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
7e68804fa3 qconfig: Move testing into a separate function
Reduce the size of main() by putting this code into its own function.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
fd35fbe222 qconfig: Move converting config args to the top
Move this check to the top, so it happens always. There is no harm to
doing this earlier and it separates the setup from actual program logic.

Update the arg rather than adding a new variable, with the new variable
only created when moving or building, since it is used more heavily.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
c50b6f18f3 qconfig: Move checking directory to the top
Move this check to the top, so it happens always. The tool should be
run from the U-Boot source directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
035ecb44ec qconfig: Move getting the colour to where it is needed
Move this assignment down to just above where it is needed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
382c662c82 qconfig: Move arg checking to the top of main()
Check for 'test' as one of the possible operations for this tool,
moving the check above the implementation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
fd385b3b01 qconfig: Move arg checking a little higher
Check for scan_source as one of the possible operations for this tool,
moving the check above the scan_source implementation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
a056c4272c qconfig: Move arg parsing into a separate function
Reduce the size of main() by putting this code into its own function.
For now the parser object needs to be returned too.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
495e58c920 qconfig: Rename the doc link
This was missed during the renaming of the tool. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: ea4d6dead37 ("moveconfig: Rename the tool to qconfig")
2024-07-26 08:01:06 -06:00
Simon Glass
0e03fb1848 qconfig: Correct format string in do_imply_config()
One of the strings was converted incorrectly. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 1bd43060b3e ("moveconfig: Use f strings where possible")
2024-07-26 08:01:06 -06:00
Simon Glass
f876e96f4c qconfig: Tidy up some pylint warnings
Reduce the number of warnings in this file a little bit.

Add my own name to the copyright message.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
0a0c124008 qconfig: Make KconfigScanner a function
This doesn't have any methods so is not good as a class. Make it a
function instead, to keep pylint happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
deedf65306 qconfig: Drop the try_expand() function
This is not used anymore, so drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:06 -06:00
Simon Glass
61d555d8a7 qconfig: Fix pylint error in read_database()
Fix this error by initing the variable before the loop:

tools/qconfig.py:880:22: E0606: Possibly using variable 'defconfig'
   before assignment (possibly-used-before-assignment)

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-26 08:01:05 -06:00
Marek Vasut
b51be8ebcb tools: Remove duplicate newlines
Drop all duplicate newlines. No functional change.

Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
2024-07-15 12:12:18 -06:00
Tom Rini
e13fcae3fc Revert "buildman: Always use the full path in CROSS_COMPILE"
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.

This reverts commit 6c0a3cf75f72370deec3ee516a9dd377397af207.

Signed-off-by: Tom Rini <trini@konsulko.com>
2024-07-08 09:12:34 -06:00
Alexander Dahl
6074f6e857 mkimage: Allow 'auto-conf' signing of scripts
U-Boot configured for verified boot with the "required" option set to
"conf" also checks scripts put in FIT images for a valid signature, and
refuses to source and run such a script if the signature for the
configuration is bad or missing.  Such a script could not be packaged
before, because mkimage failed like this:

    % tools/mkimage -T script -C none -d tmp/my.scr -f auto-conf -k tmp -g dev -o sha256,rsa4096 my.uimg
    Failed to find any images for configuration 'conf-1/signature'
    tools/mkimage Can't add hashes to FIT blob: -1
    Error: Bad parameters for FIT image type

This is especially unfortunate if LEGACY_IMAGE_FORMAT is disabled as
recommended.

Listing the script configuration in a "sign-images" subnode instead,
would have added even more complexity to the already complex auto fit
generation code.

Signed-off-by: Alexander Dahl <ada@thorsis.com>
2024-07-05 13:57:01 -06:00
Simon Glass
0318126236 u_boot_pylib: Use correct coverage tool within venv
When running within a Python venv we must use the 'coverage' tool (which
is within the venv) so that the venv packages are used in preference to
system packages. Otherwise the coverage tests run in a different
environment from the normal tests and may fail due to missing packages.

Handle this by detecting the venv and changing the tool name.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
6c0a3cf75f buildman: Always use the full path in CROSS_COMPILE
The feature to set the toolchain path does not seem to be needed. It
causes problems with venv (see [1]). Let's remove it.

Add some tests while we are here.

It does not look like any docs changes are needed for this.

[1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
2024-07-03 07:36:33 +01:00
Simon Glass
57abd7c549 buildman: Fix a few typos in toolchain code
Fix 'Thie' and capitalise 'unicode'.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2024-07-03 07:36:33 +01:00
Simon Glass
fa77b51072 buildman: Add python3-pycryptodome
This is used by some Binman entry types, so add it to allow more tests
to pass.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2024-07-03 07:36:33 +01:00
Simon Glass
e70bac90ff buildman: Add python3-coverage
Add this package so we can run code-coverage tests for Binman.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2024-07-03 07:36:33 +01:00
Simon Glass
5d679f801d buildman: Add a way to limit the number of buildmans
Buildman uses all available CPUs by default, so running more than one or
two concurrent processes is not normally useful.

However in some CI cases we want to be able to run several jobs at once
to save time. For example, in a lab situation we may want to run a test
on 20 boards at a time, since only the build step actually takes much
CPU.

Add an option which allows such a limit. When buildman starts up, it
waits until the number of running processes goes below the limit, then
claims a spot in the list. The list is maintained with a temporary file.

Note that the temp file is user-specific, since it is hard to create a
locked temporary file which can be accessed by any user. In most cases,
only one user is running jobs on a machine, so this should not matter.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
8941477e02 buildman: Add a flag to force mrproper on failure
When a file is removed by a commit (e.g. include/common.h yay!) it can
cause incremental build failures since one of the dependency files from
a previous build may mention the file.

Add an option to run 'make mrproper' automatically when a build fails.
This can be used to automatically resolve the problem, without always
adding the large overhead of 'make mrproper' to every build.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
7e93bd30b1 buildman: Avoid rebuilding when --mrproper is used
When this flag is enabled, 'make mrproper' is always used when
reconfiguring, so there is no point in doing it again.

Update this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
3187da4bea buildman: Make mrproper an argument to run_commit()
Pass this in so the caller can change it independently of the member
variable.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
3f972a4655 buildman: Make mrproper an argument to _config_and_build()
Pass this in so the caller can change it independently of the member
variable.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
49b158a492 buildman: Make mrproper an argument to _reconfigure()
Pass this in so the caller can change it independently of the member
variable.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
0f851e2341 binman: Make Intel ME default to position 0x1000
This cannot ever go at offset 0 since the descriptor is there. Use a
better offset for the ME, as used by link and coral, for example.

This matters when we start using assumed sizes for missing blobs.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
404936e573 binman: Support an assumed size for missing binaries
Binman has a the useful feature of handling missing external blobs
gracefully, including allowing them to be missing, deciding whether the
resulting image is functional or not and faking blobs when this is
necessary for particular tools (e.g. mkimage).

This feature is widely used in CI. One drawback is that if U-Boot grows
too large to fit along with the required blobs, then this is not
discovered until someone does a 'real' build which includes the blobs.

Add a 'assume-size' property to entries to allow Binman to reserve a
given size for missing external blobs.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
d552564e1a binman: Update the entrydocs header
Reduce the length of the underline for this header, to match the
heading itself.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
638aa113e0 binman: ti: Regenerate entry docs
Correct formatting errors in the documentation.

Regenerate the entries.rst file to include this recent addition.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
e1b5947705 binman: Regenerate nxp docs
Regenerate the entries.rst file to include this recent addition.

Note that more docs are needed here, to actually describe the entry
type.

Note also that the entry type needs Binman tests added.

Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Simon Glass
e344f836fe binman: efi: Correct entry docs
Somehow the class documentation has got out of sync with the generated
entries.rst file. Regenerating it causes errors, so correct these and
regenerate the entries.rst file.

Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 809f28e7213 ("binman: capsule: Use dumped capsule header...")
2024-07-03 07:36:33 +01:00
Brandon Maier
a148875093 tools: patman: fix deprecated Python ConfigParser methods
The method `ConfigParser.readfp()` is marked deprecated[1].

In Python 3.12 this method have been removed, so replace it with
`ConfigParser.read_file()`.

[1] https://docs.python.org/3.11/library/configparser.html#configparser.ConfigParser.readfp

Signed-off-by: Brandon Maier <brandon.maier@collins.com>
CC: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Brandon Maier
698b60a6a2 tools: binman: fix deprecated Python ConfigParser methods
The method `ConfigParser.readfp()` is marked deprecated[1].

In Python 3.12 this method have been removed, so replace it with
`ConfigParser.read_file()`.

[1] https://docs.python.org/3.11/library/configparser.html#configparser.ConfigParser.readfp

Signed-off-by: Brandon Maier <brandon.maier@collins.com>
CC: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Brandon Maier
357bfca5e6 tools: binman: fix deprecated Python unittest methods
The methods `unittest.assertEquals()` and
`unittest.assertRegexpMatches()` are marked deprecated[1].

In Python 3.12 these aliases have been removed, so do a sed to replace
them with their new names.

[1] https://docs.python.org/3.11/library/unittest.html#deprecated-aliases

Signed-off-by: Brandon Maier <brandon.maier@collins.com>
CC: Simon Glass <sjg@chromium.org>
CC: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2024-07-03 07:36:33 +01:00
Sean Anderson
18de1afd48 patman: Add a tag for when a patch gets added to a series
When a patch is added to a series after the initial version, there are no
changes to note except that it is new. This is typically done to suppress
the "(no changes in vN)" message. It's also nice to add a change to the
cover letter so reviewers know there is an additional patch. Add a tag to
automate this process a bit.

There are two nits with the current approach:

- It favors '-' as a bullet point, but some people may prefer '*' (or
  something else)
- Tags (e.g. 'patman: ' in 'patman: foo bar') are not stripped. They are
  probably just noise in most series, but they may be useful for treewide
  series to distinguish 'gpio: frobnicate' from 'reset: frobnicate', so
  I've left them in.

Suggestions for the above appreciated.

Suggested-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
2024-07-03 07:36:32 +01:00
Sean Anderson
b4f73931ed patman: Add Commit-cc as an alias for Patch-cc
Most tags referring to commits (or patches) are named Commit-something. The
exception is Patch-cc. Add a Commit-cc alias so we can use whichever one is
convenient.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
2024-07-03 07:36:32 +01:00