Merge branch '100-create-documentation-for-automated-testing' into 'dev'

Resolve "Create documentation for automated-testing"

See merge request fusiondirectory/dev-manual!157

source/fusiondirectory/development/automatedtesting/index.rst

+Testing
+========
+
+The base project for testing is called `automated-testing`. Here all source code for the tests and test framework is placed.
+
+Prerequisites
+-------------
+
+In order for the existing tests to run properly, some prerequisites are needed:
+
+* **Selenium server**: We are using Selenium as a running server for our tests. The latest version can be downloaded from the official page: `<https://www.selenium.dev/downloads/>`_.
+* **Java**: Tests are written exclusively using Java 17. The version can be installed using `<https://computingforgeeks.com/install-oracle-java-openjdk-on-debian-linux/>`_ as a reference for Debian-based systems.
+* **Geckodriver**: Tests are written for the Firefox browser, for which the Geckodriver is needed. We are using version 0.35, which can be downloaded from `<https://github.com/mozilla/geckodriver/releases>`_.
+* **VNU Jar**: Some tests need the VNU jar to be available on the machine. This can be downloaded using `<https://www.npmjs.com/package/vnu-jar>`_ as a reference.
+* **DSA server**: Some tests need a DSA server to be available on the machine. This is done by modifying the ``orchestrator.conf`` file from your configuration path to have ``tester`` as both username and password.
+
+.. _setup:
+
+Setup
+-----
+
+In order to run the tests properly, follow these steps:
+
+1. Clone the `automated-testing` project.
+2. Run ``cd ./automated-testing/qa``.
+3. Edit ``qa/src/resources/testsConfig.ini`` file to match your data:
+
+   * Modify the LDAP-related values.
+   * Find the VNU jar path on your device and paste it under the correct variable name.
+   * Make sure that the folders for downloads and screenshots exist.
+   * Do not modify the name of the variables, as the tests will not work otherwise.
+
+4. Save the file modifications.
+5. Run ``./gradlew clean`` (if using a Linux-based system) or ``gradle clean`` (if using Windows).
+6. Run the tests:
+
+   * To run all the tests, execute ``./gradlew test`` (this will take some time as the code base is large, but you will see the test progress).
+   * To run specific tests, execute ``./gradlew test --tests "..."``, where ``...`` is the path to your test starting from ``qa/src/test/java``. This can be:
+     * A full subfolder (e.g., ``org.fd.tests.core.*`` to run all tests in the core folder).
+     * A specific class of tests (e.g., ``org.fd.tests.core.LoginTest`` to run all tests from the ``LoginTest`` class).
+     * A specific test (e.g., ``org.fd.tests.core.LoginTest.testGoodLogin`` to only run ``testGoodLogin`` from the ``LoginTest`` class).
+
+7. After tests complete, you will see:
+
+   * ``BUILD SUCCESSFUL`` (in green) if all tests passed.
+   * ``BUILD FAILED`` (in red) if any test failed.
+   * In both cases, the test results will display:
+
+     * ``PASSED`` for successful tests.
+     * ``FAILED`` for failed tests, along with a short message (for troubleshooting, see :ref:`troubleshooting`).
+
+.. _troubleshooting:
+
+Troubleshooting
+---------------
+
+For every failed test, we have implemented features in our framework to aid troubleshooting:
+
+* If the test is running in a visual interface, we take a screenshot of the last visual screen of the driver (saved in the ``screenshots`` folder defined in ``qa/src/resources/testsConfig.ini``).
+* Logs are saved in a file in the same ``screenshots`` folder. These logs help track the workflow of the test and identify where and why it crashed.
+
+Project Structure
+-----------------
+
+All important modifiable parts of the code base are inside the ``qa`` folder:
+
+* ``config`` folder: Defines lint rules in XML format, checked by the pipeline whenever code is pushed to GitLab.
+* ``build.gradle`` file: Defines the needed plugins for this project.
+* ``src/test`` folder: Contains all the test code:
+
+  * ``resources`` folder: Stores additional files:
+
+    * ``testsConfig.ini``: The initial configuration for the tests, should be modified as stated in :ref:`setup`.
+    * ``ldifs`` folder: Contains LDIF files inserted into the LDAP server before the tests run.
+    * General rule: All additional non-Java files required for tests should be placed here.
+
+  * ``test/org/fd`` folder: Contains all test classes:
+
+    * ``Utils.java``: A class with static fields, mainly Java translations of ``testsConfig.ini`` variables.
+    * ``LdapConnection.java``: Defines LDAP-related actions like emptying the LDAP (``emptyLdap()``) and inserting an LDIF (``insertLdif(String filename)``).
+    * ``Assertions.java``: Contains methods for assertions, such as checking if a user is logged in (``assertLoggedIn(String username)``).
+    * ``ScreenshotTestWatcher.java``: Defines actions executed after a test completes, including distinguishing between failed, successful, and aborted tests. See :ref:`troubleshooting`.
+    * ``FusionDirectoryTestCase.java``: The main test class template in FusionDirectory. It integrates ``LdapConnection``, ``Assertions``, and ``ScreenshotTestWatcher`` with Selenium to interact with FusionDirectory’s web interface.
+
+  * ``tests`` folder: Contains explicit tests:
+
+    * ``core`` folder: Tests verifying the core functionality of FusionDirectory.
+    * ``plugins`` folder: Contains one subfolder for each FusionDirectory plugin that has tests.
+    * ``tools`` folder: Contains tests for various tools (schema manager, plugin manager, configuration manager, migration manager, and orchestrator client):
+
+      * Unlike other folders, these tests often require Unix console access rather than web interface interaction.
+      * ``CommandLineTestCase.java``: Defines methods for interacting with the command line:
+
+        * ``executeCommandWithWait(String command)``: Used when no user input is required.
+        * ``executeCommandWithoutWait(String command)``: Used when user input is required, handled through the returned ``Response`` object.
+      * ``CommandLineTestWatcher.java``: A smaller version of ``ScreenshotTestWatcher.java`` that only copies log files, as screenshots are not applicable.
+
+    * ``installation`` folder: Contains a single test verifying the installation page functionality.
+
+How to Write a Test
+-------------------
+
+Before writing a test, determine where it should be placed by asking, *What am I testing?* If the answer is a plugin ``abc``, place the test in the ``plugins/abc`` folder. Then:
+
+1. Create a class in the appropriate folder, ensuring the name ends in ``Test``.
+2. In the class constructor, define the ``initLdifs`` array with required LDIF files, or leave it empty if none are needed.
+3. If altering FusionDirectory-related concepts, revert changes before the test ends or use an ``@AfterEach`` annotated method to clean up automatically.
+4. When fetching web interface elements, check ``FusionDirectoryTestCase.java`` for existing methods before writing new ones to maintain clean tests and ensure easy maintenance.
+5. Annotate test methods with ``@Test`` (or similar annotations like ``@RepeatedTest(n)`` or ``@ParameterizedTest(methodName)``) for Gradle compatibility.
+
+Before pushing to GitLab, perform these checks:
+
+* Run ``./gradlew checkstyleTest`` and ``./gradlew spotbugsTest`` and fix any issues.
+* Run the tests in your virtual machine and ensure they pass.
+* Run the tests in a Bullseye Docker image and an Ubuntu Docker image to verify cross-platform compatibility.
+
+By following these steps, we minimize failed pipelines and expedite code merging, ensuring tests facilitate development rather than hinder it. Reliable tests are key to this process.
+

source/fusiondirectory/development/index.rst

@@ -9,4 +9,5 @@ Development
    codingstyle/index.rst
    writeplugin/index.rst   
    api/index.rst
+   automatedtesting/index.rst
    documentation/index.rst