Below is a procedure to import a bunch of random users in the web interface.

The scripts of this folder can also be used otherwise (see internal wiki).


# Tools for adding devices, users and agents to XiVO

Devices and user can be inserted from XiVO server or from another computer. Before using another computer,
open `/etc/fail2ban/jail.conf` and set it's IP to be ignored. Then restart fail2ban.

CSV files with users should not have more than 250 users, otherwise timeout error may appear and users will not be
imported.


**NOTICE: The old steps mentioned after this sections are DEPRECATED. Use the steps below to add users with devices in your xivo.**

0. Prepare python3 virtual environment
    - create virtual environment using `virtualenv <virtualenv-name>`
    - source the new virtual env. (from current working dir) using `source venv/bin/activate`
    - install dependencies `pip install requests`

1. Setup your XiVO
    - **Add webservice access**: Configuration > Management > Web Services Access > Add
        - Login: `ws_user`, pass = `ws_pass` and ACL = `confd.#`
        Note: For pairing lines with devices an ACL containing each device & line id is needed. That is why such ACL is granted in this case.
    - **Add a context**: Service > IPBX > IPBX Configuration > Contexts > Add
        - Context type: `internal`, Include sub-context: `Outcalls (to-extern)` (for outgoing calls)
        - Add user number range

2. Setup device plugins on XiVO
    - **Add plugin repository**: Configuration > Provisioning > General > General settings, set URL & save.
    Note: choose any of the URLs to install plugins, then switch to another URL to get different plugins.
        - http://provd.xivo.solutions/plugins/2/addons/testing/
        - http://provd.xivo.solutions/plugins/2/addons/stable/
        - http://provd.xivo.solutions/plugins/2/archive/
        - http://provd.xivo.solutions/plugins/2/stable/
        - http://provd.xivo.solutions/plugins/2/testing/
    - **Choose vendor/models of plugins**: Configuration > Provisioning > Plugins > Install plugin vendor/model
    - **Install plugins**: Configuration > Provisioning > Plugins > Edit plugin vendor/model > Install plugins

3. Prepare configuration file `config.json`
    - Copy from `config.json.template` (!DO NOT INCLUDE THE COPIED FILE INTO VERSIONING SYSTEM!)
    - Edit `plugin_config` in `config.json` and add vendors, models and plugins that reflect your plugin setup on your XiVO
        - First level: Vendor (object), eg. Yealink
        - Second level: Model (array), eg. xivo-yealink-v85
        - Third level: Plugins (element of array) eg. T31P
    - The other parts are our variables, so let's take a closer look :
        - "create_users" : This variable determines if we want to create users or not (true or false)
        - "create_agents" : This variable determines if you want to create agents associated with users (true or false)
        - "populate_queue" : This variable determines if you want to populate a queue with agents (true or false)
    - Note : You can play with these 3 parameters according to your need, example : You have users already existing but need them to become agents, disable - "create_users" while keeping "create_agents" enabled. Example 2 : You have agents but just want to populate the queue with them disable both "create_users" - and "create_agents" while keeping "populate_xivo" enabled.
        - "ws_user" : This is your Web Services Access login that you created earlier
        - "ws_pass" : This is your Web Services Access password that you created
        - "url_prefix" : This variable is the url used for api calls, it is of the form "https://<xivo_ip>:9486/1.1"
        - "user_tag" : Firstname of user
        - "line_type" : This variable determines what type of line the users you created need (sip or webrtc)
        - "first_exten" : This is the number of the first user and agent you create
        - "password" : This is the password for the user CTI access
        - "count" : Number of users and agents you create, their numbers is incremeted (ex : if your "first_exten"=1000 and your "count"=50, you will create users and - agents with numbers from 1000 to 1049)
        - "context" : The context in which you create your users and agents
        - "mds" : The mds your users and agents are associated with
        - "user_tag_increment" : A user naming value
        - "user_tag_first" : A number that increments and is in the user's firstname and lastname
        - "agents_group" : The group in which your agents belong
        - "queue_id" : The id of the queue you want to populate with agents
        - "delete_everything" : Set to true or false, if set to true, you won't create anything but delete all resources on the Xivo 
4. Run `python populate_xivo.py` or `python3 populate_xivo.py` to populate XiVO with what you want : Create users, lines, devices and agents, associate users with agents and populate queues with agents. If needed change the config.json, to add another kind of users.


# Here is a closer look at the different scripts

## common.py

Here we have the api_call function that calls apis and will be used in most of our functions

## populate_xivo.py

This file is used to populate the Xivo via its functions and the functions that are defined in the other files
- First, it imports all needed packages and functions
- Set up the config file path (config.json)
- set up the configuration via loading the config.json file to get variables and creating a basic authentication with the variables "ws_user" and "ws_pass"
- If the variable "delete_everything" is set to "true" :
    - It starts the begin_deletion function to delete everything on the Xivo
- Else :
    - If "create_users" is true we do the following :
    - If "line_type" is "webrtc" :
        - Create users and then upload those users to Xivo
    - If "line_type" is "sip" :
        - Create users, disable live_reload_configuration and upload those users to Xivo
        - Extract the line ids of those users
        - Create devices and then upload those devices to Xivo
        - Extract their devices ids
        - Assiciate the user lines with those devices
    - If "create_agents" is true :
        - Create agents and associate them to users
    - If "populate_queue" is true :
        - Populate a queue via its id with the agents

## user_generator.py

This file contains the create_user function and some steps
- First, the create_users function create an empty array
- Then, we will loop through our "count" each time generating a name with some different id and appending to the appending to the array user's values
- After some write operations, we return values that will be upload afterwards in the populate_xivo.py file

## device_generator.py

This file is for creating devices that will be linked to users in the popylate_xivo.py file
- First, in the create_devices function an empty array is created
- Afterwards, we loop throught our "count" and append values to the array that will be returned
- Now, the _pick_random_phone_plugin handle the plugin part for device
- Finally, the _format_mac_address handles the mac addresses of the devices

## agents_creation.py

This file is for the creation of agents and their association with users
- First, the create_agents function creates agents via a loop of api calls with payload
- Afterwards, the get_first_user_uuid retrieves the first user's uuid with the /users/export endpoint
- Then, with the first user's uuid, the get_first_user_id function retrieves the first user's id
- Now,we use the get_first_agent_id to retrieve the first agent's id
- And now, with both the user's and the agent's id we loop through all user's and give them an agent's id and now the agent's are created and have matching users
- Finally, if the "populate_queue" is set to true, the queue_populate function will loop through the agents and add them to the queue via its queue_id

## resources_deletion.py

This file contains the functions neede to clean a Xivo from its agents, users, lines, devices and extensions
- First you will get a confirmation that you want to delete the resources via the begin_deletion function
- Afterwards, you will get a list of the agents ids with the resources_list function
- Then, the delete agents function will disconnect the agents via their ids from their queues and then delete them completely
- Now, with the same resources_list function, we will get the ids of the lines
- After that, the delete_lines function will extract the devices, users and extensions ids that will be passed to the delete_resource function
- Finally, the delete_resource function will dissociate the resources (device, user and extension) from the lines and then delete them and delete the line

## agents_sql.py

This file is deprecated and not used in the automated lanch of the xivo population
- This file just creates a .sql file that contains agents and commands to update users to match those agents
- There are some configurations to be done in the begining of the file like first_user and add_users which match first_exten and count
- Once you set those values, you can lanch the script while giving it a file name as a parameter
- Then apply the containt of that file on your asterisk's database and you just added agents via sql


**NOTICE: the following steps are deprecated, but not removed for backwards compatibility and also they document the steps that are beign done in `populate_xivo.py`, which automates them.**

old notice: some steps were improved elsewhere, see scripts/demo_agents/README.md

1. Set XiVO IP
    - Open `add-device-by-mac.py` and change `XIVO_HOST`
    - Open `edit_users-add-device-by-mac.py` and change `XIVO_HOST`

2. Add a web service via XiVO webi
    - Configuration > Management > Web Services Access > Add
    - Login: `ws_user`, pass = `ws_pass` and ACL = `confd.#`

3. Add users
    - Turn off Live reload configuration (Configuration > Management > General)
    - Add context (IPBX configuration > Contexts > Add)
    - Set context type: Internal
    - Add range (Contexts > Edit > Users > Add)
    - Open `users_csv.py`
    - Edit `first_user` (first extension number), `add_users` (how many users to add) and `context`
    - Generate csv file: `python users_csv.py users.csv`
    - Import users via webi

4. Add devices by mac addresses
    - Open `add-device_csv.py`
    - Edit codec lists
    - Install all the codecs via webi
    - Edit `first_user` and `add_users`
    - Generate csv file: `python add-device_csv.py add.csv`
    - Add devices to XiVO: `python add-device-by-mac.py add.csv`

5. Associate devices with users
    - Open `edit_csv.py`
    - Edit `first_user` and `add_users`
    - Generate csv file: `python edit_csv.py edit.csv`
    - Associate devices with users: `python edit_users-add-device-by-mac.py edit.csv`

6. Add agents and associate them with users
    - Create an agents group (Call center > Agents > Add a group)
    - Open database: `sudo -u postgres psql -d asterisk`
    - Get the agents group id: `select * from agentgroup;`
    - Open `agents_sql.py`
    - Edit `first_user`, `add_users`, `context` and `agents_group_id`
    - Generate sql file: `python agents_sql.py /var/tmp/agents.sql`
    - Stop XiVO
    - Apply: `sudo -u postgres psql -d asterisk -a -f /var/tmp/agents.sql`

7. Count users which need to be updated for load test
    ```
    SELECT count(*) FROM usersip
    JOIN userfeatures ON (usersip.name = userfeatures.lastname)
    WHERE userfeatures.firstname='tester'
    AND usersip.host='dynamic';
    ```
8. If count matches, change *host* variable of the users to load tester IP
    ```
    UPDATE usersip
    SET host='192.168.29.239'
    FROM userfeatures
    WHERE usersip.name = userfeatures.lastname
    AND userfeatures.firstname='tester'
    AND usersip.host='dynamic';
    ```
9. In case that you need to create agents from new users use `agents_sql.py` to generate a sql command that will pair your users with agent profiles.
    - First edit the `agents_sql.py` according to values from `config.json`.
        - `first_user` should be same as `first_exten`.
        - `add_users` should be same as `count`.
        - `context` should be same as `context`.
    - Then run the script with arguments:
        - First argument `agents_group_id` which can be retrieved from `xivo-db` using `psql -U asterisk -c 'SELECT groupid FROM agentgroup WHERE name = "YOUR_AGENT_GROUP_NAME"'`.
        - Second argument `filename` for the output sql file.
    - Lastly take the output sql file and run it's contents in `xivo-db` to finish agent creation.
