feat(*): Add AppSec support for LAPI (#28)

* feat(appsec): Add getAppSecRemediation method for LapiRemediation

* style(*): Pass through code format tools

* feat(appsec): Handle appsec fallback on timeout

* test(appsec): Add test for appsec_fallback_remediation in case of timeout

* style(*): Pass through code format tools

.github/workflows/markdown.yml → .github/workflows/doc-links.yml

@@ -1,20 +1,25 @@
+name: Documentation links
+
 on:
   workflow_dispatch:
-
-name: Markdown files test and update
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
 
 permissions:
-  contents: write
-  pull-requests: write
+  contents: read
 
 jobs:
-  markdown-test-and-update:
-    name: Markdown files test and update
+  markdown-test-links:
+    name: Markdown test links
     runs-on: ubuntu-latest
     steps:
 
       - name: Clone sources
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           path: extension
 
@@ -33,13 +38,4 @@ jobs:
           gem install awesome_bot
           cd extension
           awesome_bot --files README.md --allow-dupe --allow-redirect --allow 401 --skip-save-results --white-list ddev.site --base-url http://localhost:8080/
-          awesome_bot docs/*.md --allow-dupe --allow-redirect --allow 401 --skip-save-results --white-list crowdsec.net/v2,ddev.site,your-lapi-url,https://crowdsec --base-url http://localhost:8080/docs/
-
-      - name: Generate table of contents
-        uses: technote-space/toc-generator@v4
-        with:
-          MAX_HEADER_LEVEL: 5
-          COMMIT_NAME: CrowdSec Dev Bot
-          TARGET_PATHS: 'docs/*.md'
-          CHECK_ONLY_DEFAULT_BRANCH: true
-          CREATE_PR: true
+          awesome_bot docs/*.md --allow-dupe --allow-redirect --allow 401 --skip-save-results --white-list crowdsec.net/v2,ddev.site,your-lapi-url,your-appsec-url,https://crowdsec,http://crowdsec --base-url http://localhost:8080/docs/

.github/workflows/release.yml

@@ -43,7 +43,7 @@ jobs:
           echo "VERSION_NUMBER=$(echo ${{ github.event.inputs.tag_name }} | sed 's/v//g' )" >> $GITHUB_ENV
 
       - name: Clone sources
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Check version ${{ env.VERSION_NUMBER }} consistency in files
         # Check src/Constants.php and CHANGELOG.md

.github/workflows/unit-and-integration-test.yml

@@ -61,13 +61,18 @@ jobs:
           ddev get julienloizelet/ddev-crowdsec-php
 
       - name: Start DDEV
-        run: |
-          ddev start
+        uses: nick-fields/retry@v3
+        with:
+          timeout_minutes: 5
+          max_attempts: 3
+          shell: bash
+          command: ddev start
 
       - name: Some DEBUG information
         run: |
           ddev --version
           ddev exec php -v
+          ddev exec -s crowdsec cscli version 
 
       - name: Clone sources
         uses: actions/checkout@v4

CHANGELOG.md

@@ -10,8 +10,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 The [public API](https://semver.org/spec/v2.0.0.html#spec-item-1)  of this library consists of all public or protected methods, properties and constants belonging to the `src` folder.
 
+As far as possible, we try to adhere to [Symfony guidelines](https://symfony.com/doc/current/contributing/code/bc.html#working-on-symfony-code) when deciding whether a change is a breaking change or not.
+
+
+---
+
+## [3.4.0](https://github.com/crowdsecurity/php-remediation-engine/releases/tag/v3.3.0) - 2024-??-??
+[_Compare with previous release_](https://github.com/crowdsecurity/php-remediation-engine/compare/v3.3.0...v3.4.0)
+
+### Added
+
+- Add AppSec requests support with `getAppSecRemediation` public method in `LapiRemediation` class
+
+
+### Changed
+
+- Update `crowdsec/common` dependency to `v2.3.0`
+- Update `crowdsec/capi-client` dependency to `v3.2.0`
+- Update `crowdsec/lapi-client` dependency to `v3.3.0`
+
+
 ---
 
+
 ## [3.3.0](https://github.com/crowdsecurity/php-remediation-engine/releases/tag/v3.3.0) - 2023-12-14
 [_Compare with previous release_](https://github.com/crowdsecurity/php-remediation-engine/compare/v3.2.0...v3.3.0)
 

composer.json

@@ -45,9 +45,9 @@
         "ext-json": "*",
         "symfony/config": "^4.4.27 || ^5.2 || ^6.0",
         "symfony/cache": "^5.4.11|| ^6.0.11",
-        "crowdsec/common": "^2.2.0",
-        "crowdsec/capi-client": "^3.1.0",
-        "crowdsec/lapi-client": "^3.2.0",
+        "crowdsec/common": "^2.3.0",
+        "crowdsec/capi-client": "^3.2.0",
+        "crowdsec/lapi-client": "^3.3.0",
         "monolog/monolog": "^1.17 || ^2.1",
         "mlocati/ip-lib": "^1.18",
         "geoip2/geoip2": "^2.13.0"

docs/DEVELOPER.md

@@ -17,14 +17,9 @@
     - [Unit test](#unit-test)
     - [Integration test](#integration-test)
     - [Coding standards](#coding-standards)
-      - [PHPCS Fixer](#phpcs-fixer)
-      - [PHPSTAN](#phpstan)
-      - [PHP Mess Detector](#php-mess-detector)
-      - [PHPCS and PHPCBF](#phpcs-and-phpcbf)
-      - [PSALM](#psalm)
-      - [PHP Unit Code coverage](#php-unit-code-coverage)
 - [Commit message](#commit-message)
   - [Allowed message `type` values](#allowed-message-type-values)
+- [Update documentation table of contents](#update-documentation-table-of-contents)
 - [Release process](#release-process)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -118,6 +113,13 @@ Run:
 ddev composer update --working-dir ./my-code/remediation-engine
 ```
 
+For advanced usage, you can create a `composer-dev.json` file in the `my-code/remediation-engine` folder and run:
+
+```bash
+ddev exec COMPOSER=composer-dev.json composer update --working-dir ./my-code/remediation-engine
+```
+
+
 #### Unit test
 
 ```bash
@@ -144,6 +146,12 @@ In order to use these, you will need to work with a PHP version >= 7.4 and run f
 ddev composer update --working-dir=./my-code/remediation-engine/tools/coding-standards
 ```
 
+For advanced usage, you can create a `composer-dev.json` file in the `my-code/remediation-engine/tools/coding-standards` folder and run:
+
+```bash
+ddev exec COMPOSER=composer-dev.json composer update --working-dir ./my-code/remediation-engine/tools/coding-standards 
+```
+
 ##### PHPCS Fixer
 
 We are using the [PHP Coding Standards Fixer](https://cs.symfony.com/)
@@ -263,6 +271,23 @@ chmod +x .git/hooks/commit-msg
 - style (formatting; no production code change)
 - test (adding missing tests, refactoring tests; no production code change)
 
+## Update documentation table of contents
+
+To update the table of contents in the documentation, you can use [the `doctoc` tool](https://github.com/thlorenz/doctoc).
+
+First, install it:
+
+```bash
+npm install -g doctoc
+```
+
+Then, run it in the documentation folder:
+
+```bash
+doctoc docs/* --maxlevel 4  
+```
+
+
 ## Release process
 
 We are using [semantic versioning](https://semver.org/) to determine a version number. To verify the current tag, 

docs/USER_GUIDE.md

@@ -15,24 +15,11 @@
   - [Capi Remediation](#capi-remediation)
     - [Instantiation](#instantiation)
     - [Features](#features-1)
-      - [Retrieve fresh decisions from CAPI](#retrieve-fresh-decisions-from-capi)
-      - [Get remediation for an IP](#get-remediation-for-an-ip)
-      - [Clear cache](#clear-cache)
-      - [Prune cache](#prune-cache)
     - [Stream mode and example scripts](#stream-mode-and-example-scripts)
   - [Lapi Remediation](#lapi-remediation)
     - [Instantiation](#instantiation-1)
     - [Features](#features-2)
-      - [Get Decisions stream list from LAPI](#get-decisions-stream-list-from-lapi)
-      - [Get remediation for an IP](#get-remediation-for-an-ip-1)
-      - [Clear cache](#clear-cache-1)
-      - [Prune cache](#prune-cache-1)
     - [Example scripts](#example-scripts)
-      - [Get decisions stream](#get-decisions-stream)
-      - [Get remediation for an IP](#get-remediation-for-an-ip-2)
-      - [Get remediation for an IP using geolocation](#get-remediation-for-an-ip-using-geolocation)
-      - [Clear cache](#clear-cache-2)
-      - [Prune cache](#prune-cache-2)
 - [CAPI remediation engine configurations](#capi-remediation-engine-configurations)
   - [Remediation priorities](#remediation-priorities)
   - [Remediation fallback](#remediation-fallback)
@@ -47,6 +34,7 @@
   - [Redis cache DSN](#redis-cache-dsn)
   - [Memcached cache DSN](#memcached-cache-dsn)
   - [Cache tags](#cache-tags)
+  - [AppSec url](#appsec-url)
 - [Helpers](#helpers)
   - [Origins count](#origins-count)
 
@@ -75,6 +63,7 @@ This kind of action is called a remediation and can be:
     - Use the cached decisions for CAPI and for LAPI in stream mode
     - For LAPI in live mode, call LAPI if there is no cached decision
     - Use customizable remediation priorities
+  - Determine AppSec (LAPI) remediation for a given request 
 
 - Overridable cache handler (built-in support for `Redis`, `Memcached` and `PhpFiles` caches)
 
@@ -190,6 +179,8 @@ Unlike Memcached and Redis, there is no PhpFiles pruning mechanism that automati
 Thus, if you are using the PhpFiles cache, you should use this method.
 
 
+
+
 #### Stream mode and example scripts
 
 The CAPI remediation engine is intended to work asynchronously: this is what we call the `stream mode`: 
@@ -255,6 +246,7 @@ $logger = new FileLog(['debug_mode' => true]);
 $clientConfigs = [
     'auth_type' => 'api_key',
     'api_url' => 'http://your-lapi-url:8080',
+    'app_sec_url' => 'http://your-appsec-url:7422',
     'api_key' => '****************',
 ];
 $lapiClient = new Bouncer($clientConfigs, null, $logger);
@@ -331,6 +323,21 @@ $remediationEngine->pruneCache();
 Unlike Memcached and Redis, there is no PhpFiles pruning mechanism that automatically removes expired items.
 Thus, if you are using the PhpFiles cache, you should use this method.
 
+##### Get AppSec remediation for a request
+
+To retrieve an AppSec decision, you can do the following call:
+
+```php
+$remediationEngine->getAppSecRemediation($headers, $rawBody);
+```
+
+The `$headers` parameter is an array containing the headers of the forwarded request and some required headers for the AppSec decision.
+
+The `$rawBody` parameter is optional and must be used if the forwarded request contains a body. It must be a string.
+
+Please see the [CrowdSec AppSec documentation](https://docs.crowdsec.net/docs/appsec/intro) for more details.
+
+
 #### Example scripts
 
 You will find some ready-to-use php scripts in the `tests/scripts` folder. These scripts could be useful to better
@@ -412,6 +419,20 @@ php tests/scripts/prune-cache-lapi.php <BOUNCER_KEY> <LAPI_URL>
 php tests/scripts/prune-cache-lapi.php c580ebdff45da6e01415ed0e9bc9c06b  https://crowdsec:8080
 ```
 
+##### Get AppSec remediation
+
+###### Command usage
+
+```php
+php tests/scripts/get-remediation-appsec.php <APPSEC_URL> <IP> <URI> <HOST> <VERB> <API_KEY> <USER_AGENT> <HEADERS_JSON> [<RAW_BODY>]
+```
+
+###### Example usage
+
+```bash
+php tests/scripts/get-appsec-remediation http://crowdsec:7422  172.0.0.24 /login example.com POST c580eb*********de541 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0' '{"Content-Type":"application/x-www-form-urlencoded","Accept-Language":"en-US,en;q=0.5"}' 'class.module.classLoader.resources.'
+```
+
 
 ## CAPI remediation engine configurations
 
@@ -639,24 +660,39 @@ Beware that there is a caveat with Symfony tagged caching and Redis: it doesn't
 Cache tags is not supported for the provided Memcached cache.
 
 
+### AppSec url
+
+If you want to use the AppSec feature, you need to set the `app_sec_url` setting:
+
+```php
+$configs = [
+...
+'app_sec_url' => https://your-appsec-url:7422
+...
+];
+```
+
+This setting is not required and is `http://localhost:7422` by default.
+
 ## Helpers
 
 ### Origins count
 
-In order to have some metrics, we store in cache the number of calls to the `getIpRemedation` method while 
+In order to have some metrics, we store in cache the number of calls to the `getIpRemedation` and `getAppSecRemediation` methods while 
 separating the counters by origin of the final remediation. 
 
 The `getOriginsCount` helper method returns an array whose keys are origins and values are the counter associated to 
-the origin. When the remediation is a `bypass` (i.e. no active decision for the tested IP), we set the origin as 
-`clean`.
+the origin. When the retrieved remediation is a `bypass` (i.e. no active decision for the tested IP or no ban returned by AppSec for the tested request), we set the origin as `clean` or `clean_appsec`.
 
 ```php
 /** @var $remediation \CrowdSec\RemediationEngine\AbstractRemediation */
 $originsCount = $remediation->getOriginsCount();
 
 /*$originsCount = [
-    'clean' => 150, 
+    'appsec' => 6,
+    'clean' => 150,
+    'clean_appsec' => 2, 
     'capi' => 28,
-    'lists' => 16
+    'lists' => 16,
 ]*/
 ```