Verified Commit 050aa330 authored by dockx thibault's avatar dockx thibault
Browse files

:rocket: feat(extractor) add extractor task documentation and client command

parent 658102df
No related merge requests found
Showing with 141 additions and 1 deletion
+141 -1
Extractor Task Execution
========================
.. _extractor-task-execution-label:
Once the task is configured, execute it using the **fusiondirectory-orchestrator-client** tool. You have two execution options:
.. code-block:: bash
fusiondirectory-orchestrator-client --extract
.. note::
**Recommendation** automate this process within a CRON job.
.. note::
Add the ``--verbose`` flag to the command for detailed output during execution.
.. note::
The logs are stored by default at the following location:
``/srv/orchestrator/``
.. note::
You can specify a custom path for the extracted files by adding a directory path to the ``--extract`` option. For example:
.. code-block:: bash
fusiondirectory-orchestrator-client --extract /var/log/
This will save the generated files in the specified directory instead of the default location.
File Naming Format
------------------
The generated files follow a specific naming convention to ensure clarity and avoid duplication:
``NameOfTheTask-date+hour.format``
- **NameOfTheTask**: The name you assigned to the task.
- **date**: The current date in `YYYYMMDD` format (e.g., `20250407` for April 7, 2025).
- **hour**: The hour of execution in `HH` format (e.g., `14` for 2 PM).
- **format**: The output format, such as `csv`.
For example, a task named `UserExport` executed on April 7, 2025, at 2 PM will generate a file named:
``UserExport-20250407-14.csv``
\ No newline at end of file
...@@ -10,4 +10,5 @@ Endpoints ...@@ -10,4 +10,5 @@ Endpoints
userreminder.rst userreminder.rst
audit.rst audit.rst
mails.rst mails.rst
archive.rst archive.rst
\ No newline at end of file extractor.rst
\ No newline at end of file
...@@ -23,6 +23,13 @@ Execute a Task of Type "Mail" ...@@ -23,6 +23,13 @@ Execute a Task of Type "Mail"
fusiondirectory-orchestrator-client --emails fusiondirectory-orchestrator-client --emails
Execute a Task of Type "Extractor"
""""""""""""""""""""""""""""""""""
.. code-block:: shell
fusiondirectory-orchestrator-client --extract [optional path]
Execute a Task of Type "Life Cycle" Execute a Task of Type "Life Cycle"
""""""""""""""""""""""""""""""""""" """""""""""""""""""""""""""""""""""
......
Extractor Task
==============
The **Extractor Task** plugin allows you to define a set of users, user groups, or dynamic groups to extract data into a specific format.
For now, only **CSV** is available, but additional formats will be supported in the future.
The CSV file adapts its headers based on the attributes collected among the users listed, ensuring proper formatting even if some users have undefined attributes compared to others.
.. note::
It must be used with FusionDirectory Orchestrator and requires the `--extract` option.
CSV Header Behavior
-------------------
The **Extractor Task** dynamically generates the headers of the CSV file based on the attributes collected from the selected users, user groups, or dynamic groups.
- If some users have attributes that others do not, the CSV file will adapt by including all attributes as headers, leaving empty values for undefined attributes.
- This ensures that the CSV file is properly formatted and contains all relevant data, even when the attributes vary between users.
File Generation Behavior
------------------------
The **Extractor Task** generates files based on the hour of the day. Each task execution during the same hour appends data to the same file, ensuring no duplication.
- Files are named in the format `NameOfTheTask-date+hour.format` (e.g., `MyTask-20250407-14.csv`).
- If the task is executed multiple times within the same hour, existing UIDs in the file are updated rather than duplicated.
- A new file is created for each subsequent hour if the task is executed again.
This behavior ensures efficient data management and prevents duplication while maintaining up-to-date information in the generated files.
Task Setup
----------
Creating the Task
-----------------
- Open the **Tasks** section of FusionDirectory.
- Define the task’s schedule and repetition interval.
.. image:: images/extractor-p1.png
:alt: Extractor - Task creation step 1
:width: 600px
Configuring Extractor Task
--------------------------
- **Navigate** to the **Tasks Extractor** tab.
- **Select** the output format desired (e.g., CSV).
- **List** the users, static groups, or dynamic groups requiring data extraction.
- **Optionally define** the output directory. By default, files are saved in `/srv/orchestrator`.
- **Assign** the relevant members.
.. note::
You can select either a **static group** or a **dynamic group** for greater flexibility.
.. image:: images/extractor-p2.png
:alt: Extractor - Task creation step 2
:width: 600px
Task Execution
--------------
For your configured task to be executed, you need to configure your `fusiondirectory-orchestrator-client` with the `--extract` option.
See :ref:`Extractor Task Execution <extractor-task-execution-label>` for more information.
Summary
-------
The **Extractor Task**, when configured as described, will:
- **Analyze** the selected users, groups, or dynamic groups.
- **Generate** a CSV file containing the collected attributes as headers.
- **Adapt** the headers dynamically to account for undefined attributes.
- **Ensure** no duplication by appending data to the same file during the same hour and updating existing UIDs.
.. note::
This ensures efficient data extraction and up-to-date file generation.
\ No newline at end of file
source/fusiondirectory/plugins/extractor/images/extractor-p1.png

33 KB

source/fusiondirectory/plugins/extractor/images/extractor-p2.png

36.9 KB

Extractor
=========
.. toctree::
:maxdepth: 2
extractorTask.rst
...@@ -24,6 +24,7 @@ FusionDirectory Plugins ...@@ -24,6 +24,7 @@ FusionDirectory Plugins
dsa/index dsa/index
dyngroups/index dyngroups/index
ejbca/index ejbca/index
extractor/index
fai/index fai/index
freeradius/index freeradius/index
fusioninventory/index fusioninventory/index
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment