[#136] Update Licenses Inventory tool to version v4.0.4 (#137)

* chore: Licenses Inventory - update README

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Laurent Body <laurent.body@orange.com>

* chore: Licenses Inventory - update config.ini

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Laurent Body <laurent.body@orange.com>

* chore: Licenses Inventory - update .gitignore

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

* chore: Licenses Inventory - update sources

Signed-off-by: Laurent Body <laurent.body@orange.com>
Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Laurent Body <laurent.body@orange.com>

* chore: Licenses Inventory - update tests

Signed-off-by: Laurent Body <laurent.body@orange.com>
Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Co-authored-by: Laurent Body <laurent.body@orange.com>

* chore: Licenses Inventory - update dry-run

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

* chore: Licenses Inventory - clean up

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

* chore: Licenses Inventory - clean up

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

* chore: Licenses Inventory - add details about tests

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

---------

Signed-off-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>
Signed-off-by: Laurent Body <laurent.body@orange.com>
Co-authored-by: Laurent Body <laurent.body@orange.com>
Co-authored-by: Pierre-Yves Lapersonne <pierreyves.lapersonne@orange.com>

Author: pylapp

.gitignore

@@ -4,4 +4,14 @@ toolbox/gitlab/data
 toolbox/utils/text-generator/_templates/new-GitHub-repository-contributors.fr.template.txt.result
 toolbox/utils/third-party-generator/components.csv.result
 toolbox/utils/third-party-generator/THIRD-PARTY.md.result
-toolbox/utils/third-party-generator/__pycache__
+toolbox/utils/third-party-generator/__pycache__
+toolbox/LicensesInventory/.pytest_cache
+toolbox/LicensesInventory/sources/__pycache__
+toolbox/LicensesInventory/sources/*/__pycache__
+toolbox/LicensesInventory/sources/data_to_test_main
+toolbox/LicensesInventory/sources/test_main.py
+toolbox/LicensesInventory/tests/.pytest_cache
+toolbox/LicensesInventory/tests/integrationtests/__pycache__
+toolbox/LicensesInventory/tests/integrationtests/real_data
+toolbox/LicensesInventory/tests/unittests/__pycache__
+toolbox/LicensesInventory/tests/unittests/data

CHANGELOG.md

@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased](https://github.com/Orange-OpenSource/floss-toolbox/compare/2.16.0..dev)
 
+### Added
+
+- [Licenses Inventory] Upgrade to version v4.0.4 ([#136](https://github.com/Orange-OpenSource/floss-toolbox/issues/136))
+
 ### Changed
 
 - [Utils] Default values for THIRD_PARTY generator script, shared configuration with prompt script

toolbox/LicensesInventory/README.md

@@ -2,25 +2,31 @@
 
 Table of Contents
 =================		
-   * [Licenses inventory](#licenses-inventory)			
-      * [Disclaimer](#disclaimer)			
-      * [Prerequisites](#prerequisites)			
-      * [Fill the configuration file](#fill-the-configuration-file)			
+   * [Licenses inventory](#licenses-inventory)
+      * [Disclaimer](#disclaimer)
+      * [What the tool does](#what-the-tool-does)
+      * [Prerequisites](#prerequisites)
+      * [Fill the configuration file](#fill-the-configuration-file)
       * [Run the tool](#run-the-tool)
       * [Run the tests](#run-the-tests)
+      * [Limits](#limits)
+      * [Scenarios](#scenarios)
+      * [Example of use](#example-of-use)
       * [Managed platforms and environments](#managed-platforms)
          * [Go with go.mod](#go-language)
          * [Gradle with build.gradle(.kts)](#gradle-environment)
-         * [Rust with Cargo.lock](#rust-environment)		 
+         * [Rust with Cargo.lock](#rust-environment)
          * [JavaScript / Node.js with package.json](#javascript--nodejs-environment)
          * [Swift with Package.swift](#swift--spm-environment)
-         * [Dart / Flutter with pubspec.yaml](#dart--flutter-environment)		 		 		 
+         * [Dart / Flutter with pubspec.yaml](#dart--flutter-environment) 		 
       * [Notes](#notes)
 
 # Licenses inventory
 
 _Keywords: #licenses #SPM #Gradle #Maven #NPMJS #package #Cocoapods #pubspec #gomod #Cargo #crates_
 
+The tool searches a license for each dependency found in the files to treat.
+
 ## Disclaimer
 
 *This is quite experimental feature, with results which must be verified by a human.*
@@ -30,6 +36,18 @@ _Keywords: #licenses #SPM #Gradle #Maven #NPMJS #package #Cocoapods #pubspec #go
 
 *Such caveats are about versions of components (not checked) and version names (not sure they are related to the good components)*
 
+**This is an experimental feature designed and implemented by a blind colleague, you must always keep in mind our [Code of Conduct](https://github.com/Orange-OpenSource/floss-toolbox/blob/dev/CODE_OF_CONDUCT.md) for any issues nor comments, and be benevolent and kind.
+This is mandatory.**
+
+## What the tool does
+
+1. Read the file 'config.ini' ;
+2. Read the files to treat ;
+3. Extract the dependencies from these files ; 
+4. Search in the web a license for each dependency ;
+5. Save the licenses ;
+6. Save the dependencies on error.
+
 ## Prerequisites
 
 - _Python_ version **3.7**
@@ -56,35 +74,75 @@ bash dry-run.sh
 Before to use the tools, the file 'config.ini' is at the root of the project, you have to personalize this file.
 
 For example:
-```text
+```ini
 [dependencies]
-# Where to find the package manager file above
+# Where to find the package manager files above, must be defined, target must exist
 path to parse = /absolute/path/to/project_to_test
-# The name of the package manager file to process store above
+# The name of the package manager file to process stored above, must be defined
 the filenames = go.mod
-# For outputs
+# For outputs, must be defined, target must exists
 path to store the licenses = /absolute/path/to/project_to_test-licences
+# Erros maangement if requests failed
+number of authorized successive errors = 1
 ```
 
 where:
 - `path to parse` contains the dependencies manager files
 - `the filenames` contains the names of the dependencies manager files to process
-- `path to store the licenses` points to a folder containing the result files
+- `path to store the licenses` points to a folder containing the result files prefixed by "licenses_" if license has been found or "errors_"  if an error occured (e.g. requests limits in web site, etc)
+- `number of authorized successive errors` is the number of succesive errors authorized before ignoring the next dependencies to treat
 
 ## Run the tool
 
 ```shell
 python3 sources/main.py
 ```
 
+For example, if you define some _Cargo.lock_ file to process in *the filenames* stored at *path to parse*, it will create in *path to store the licenses* a *Cargo_lock_ folder with some outputs (mainly HTML or JSON files) and a *licenses_Cargo.lock.txt* with the licenses of each component found.
+
 ## Run the tests
 
-To run integration tests:
+To run the tests (all must pass):
 
 ```shell
- python3 -m pytest tests/integrationtests/*.py
+# Integration tests some user inputs
+python3 -m pytest -s tests/integrationtests/*.py
+
+# Unit tests 
+python3 -m pytest tests/unittests/*.py
 ```
 
+To run the unit tests, you must get the assets attached as artificats to [the release you got](https://github.com/Orange-OpenSource/floss-toolbox/releases).
+For integration tests, get the *real_data* folder in the *integrationtests* folder and move it to the same folder in your project.
+For unit tests, get the *data* folder in the *unittests* folder and move it to the same folder in your project.
+
+Then you will have to update the configuration values defined in all the _config_ files of the data sets.
+Indeed, absolute paths are used, so you must look for any "🥜" and replace by the path fragments leading to the folders.
+
+## Limits
+
+The dependencies are always treated in the same order. The downloading can be aborted. For example, a website can limit the number of requests for a done duration. In this case, all the following dependencies will have the same error. For Gradle, we can limit the number of authorized errors to avoid to continue the unuseful downloadings.
+
+## Scenarios
+
+With no error:
+- the dependency is saved in the file "licenses_platform.txt"
+- the dependency is not saved in the file "errors_platform.txt"
+
+On error:
+- the dependency is not saved in the file "licenses_platform.txt"
+- the dependency is saved in the file "errors_platform.txt"
+
+## Example of use
+
+The user executes the tools. If dependencies are on error, the tools displays, for each treated platform, the number of new dependencies to treat, the number of dependencies on error and the number of duplicated (dependencies on error which are in the new dependencies)/
+The tools asks to the user to treat the dependencies on error or the new dependencies or to quit the program.
+
+If they are only new dependencies, the tools does not display the number of dependencies.
+If they are only dependencies on error, the tools does not display the number of dependencies.
+
+So, we can search licenses for dependencies which have not been treated following an error during the downloading.
+
 ## Managed platforms
 
 ### Go language

toolbox/LicensesInventory/config.ini

@@ -1,10 +1,5 @@
 [dependencies]
-
-#  Absolute path pointing to directory with the dependencies manager files
 path to parse = ...
-
-# Name of the dependencies manager files to process in { pubspec.yaml, go.mod, package.json, Cargo.lock, build.gradle, build.gradle.kts, Podfile, Package.swift }
 the filenames = ...
-
-# Absolute path pointing to directory where result file and outputs are written
-path to store the licenses = ...
+path to store the licenses = ...
+number of authorized successive errors = 1