Difference between revisions of "Automatic user configuration"
(26 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | + | On 01/01/2023 this information become outdated. [mailto:support@analysis.stfc.ac.uk IDAaaS Support Analysis team] is solely responsible for auto-configuring IDAaaS machines. The information below is remaining for references purposes and stating how it used to work in the past. | |
− | When an experiment is approved and scheduled by instrument scientist, [http://www.isis.stfc.ac.uk/user-office/ User office] assigns a user with his/her federal ID, which allows user to access ISIS network including [ | + | == Historical automatic user configuration procedure == |
+ | |||
+ | |||
+ | When an experiment is approved and scheduled by instrument scientist, [http://www.isis.stfc.ac.uk/user-office/ User office] assigns a user with his/her federal ID, which allows user to access ISIS network including [https://isis.analysis.stfc.ac.uk/workspaces/ IDAaaS virtual machines] and provides appropriate information to [Catalog service], giving the user rights to access the data through the Catalog. | ||
At some stage, special [http://icatingest2.isis.cclrc.ac.uk/excitations.txt information file], which defines the participants of the experiments and the scientists who have access to the data of this experiment is placed on the [http://icatingest2.isis.cclrc.ac.uk/excitations.txt internal server] where it is accessed and automatically processed by [https://github.com/FITLinuxSupport/analysisscripts/blob/master/Build_Mantid_config.py issicompute configuration script]. The format of the file is .json and the sample contents is provided below. | At some stage, special [http://icatingest2.isis.cclrc.ac.uk/excitations.txt information file], which defines the participants of the experiments and the scientists who have access to the data of this experiment is placed on the [http://icatingest2.isis.cclrc.ac.uk/excitations.txt internal server] where it is accessed and automatically processed by [https://github.com/FITLinuxSupport/analysisscripts/blob/master/Build_Mantid_config.py issicompute configuration script]. The format of the file is .json and the sample contents is provided below. | ||
− | The automatic isiscompute configuration script runs twice a day on [https://isiscompute.nd.rl.ac.uk/nxwebplayer isiscompue server] processing the information file, creating user accounts and appropriate RB folders and giving users permissions to access their RB folders on the basis of the information file. [mailto:alex.buts@stfc.ac.uk Alex Buts] and [https://www.facilities.rl.ac.uk/itsupport/default.aspx FBU IT support] [mailto:FITServiceDesk@stfc.ac.uk FBU IT support (Jeffs Warren)] have access and can modify this script. At the final stage this script runs [https://github.com/mantidproject/mantid/blob/master/scripts/Inelastic/Direct/ISISDirecInelasticConfig.py the Mantid configuration script for inelastic instruments], which configures user's Mantid to | + | The automatic isiscompute configuration script runs twice a day on [https://isiscompute.nd.rl.ac.uk/nxwebplayer isiscompue server] processing the information file, creating user accounts and appropriate RB folders and giving users permissions to access their RB folders on the basis of the information file. [mailto:alex.buts@stfc.ac.uk Alex Buts] and [https://www.facilities.rl.ac.uk/itsupport/default.aspx FBU IT support] [mailto:FITServiceDesk@stfc.ac.uk FBU IT support (Jeffs Warren)] have access and can modify this script. At the final stage this script runs [https://github.com/mantidproject/mantid/blob/master/scripts/Inelastic/Direct/ISISDirecInelasticConfig.py the Mantid configuration script for inelastic instruments], which configures each user's Mantid to use appropriate [[Experimental Data]], [[Instrument parameters and Instrument definitions]], necessary to reduce experimental data. It also provides users with initial version of [[Reduction scripts]], which users should modify to their requests according to the [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/ISIS_direct_reduction_script_description. description], using help of their instrument scientist if necessary. [mailto:alex.buts@stfc.ac.uk Alex Buts] is responsible for maintaining this script. |
+ | |||
+ | When initial [[Reduction scripts]] are copied to user's RB folder from the [https://github.com/mantidproject/scriptrepository/tree/master/direct_inelastic Mantid Script Repository] sample folder, script date is set to the experiment start date and the time -- to the exact midnight of the start date. Configuration script running twice a day checks this date and if it does not changed, replaces the existing script with the script, found in the Mantid Script Repository sample folder. If somebody touched the copied script, its date and time changes and [https://github.com/mantidproject/mantid/blob/master/scripts/Inelastic/Direct/ISISDirecInelasticConfig.py autoconfiguration script] stops replacing the file. | ||
− | |||
− | The workflow | + | |
+ | The diagram and the table below summarize the user creation workflow: | ||
+ | |||
+ | [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid] | ||
+ | |||
+ | === ISISOMPUTE User creation workflow === | ||
+ | |||
+ | [[File:UserConfigWorkflow.png|1000px]] | ||
+ | |||
+ | Less visual but more detailed description of the workflow is provided in the table below. The number of the row in the table corresponds to the number of the block in the diagram above. | ||
{| class="wikitable" style="color:blue; | {| class="wikitable" style="color:blue; | ||
Line 21: | Line 34: | ||
| -- | | -- | ||
| Two weeks before Cycle, Crhis closes archive on the previous cycle. Sets up new cycle | | Two weeks before Cycle, Crhis closes archive on the previous cycle. Sets up new cycle | ||
− | | Cycle Folders appear in the archive. | + | | Cycle Folders appear in the archive. |
| Chris Moreton-Smith | | Chris Moreton-Smith | ||
|- | |- | ||
| 2 | | 2 | ||
− | | | + | | --- |
− | | | + | | Somebody from the team sets up new cycle for cataloguing data. |
− | | | + | | Data are searchable through iCat. |
| Tom Griffin | | Tom Griffin | ||
|- | |- | ||
| 3 | | 3 | ||
− | | | + | | Results of the review pannel |
| On request, Instrument Scientists, schedules experiment in Scheduler application | | On request, Instrument Scientists, schedules experiment in Scheduler application | ||
| Input for scheduler script | | Input for scheduler script | ||
Line 44: | Line 57: | ||
| 5 | | 5 | ||
| Results from 3 & 4 | | Results from 3 & 4 | ||
− | | Twice a day: Scheduler | + | | Twice a day: '''Scheduler''' scripts runs processing inputs. |
− | | | + | | |
− | + | # Emails user office to create&renew FedID-s | |
− | + | # Generates [http://icatingest2.isis.cclrc.ac.uk/excitations.txt information file] containing RB number, Instrument, Cycle, visitors names & FedIDs | |
− | + | | Contact responsible from [mailto:isisdata@stfc.ac.uk data acquisition support group] | |
− | | | ||
|- | |- | ||
| 6 | | 6 | ||
Line 55: | Line 67: | ||
| Create FedID & renew the old one | | Create FedID & renew the old one | ||
| | | | ||
− | + | # User team as input for 5 | |
− | + | # E-mails to users | |
| User's office | | User's office | ||
|- | |- | ||
Line 63: | Line 75: | ||
| Twice a day: [https://github.com/FITLinuxSupport/analysisscripts/blob/master/Build_Mantid_config.py issicompute configuration script] | | Twice a day: [https://github.com/FITLinuxSupport/analysisscripts/blob/master/Build_Mantid_config.py issicompute configuration script] | ||
| | | | ||
− | + | # Creates RB folders, User folders, access rights, autoreduction configuration. | |
− | + | # Users got rights to log into isiscompute and access their experiment folders. | |
− | + | # Starts Mantid Configuration script (8) and provides it with information about FedID&instrument&RB number | |
− | + | | Jeff Warren & Alex Buts | |
− | |||
− | |||
− | |||
− | | Jeff Warren | ||
|- | |- | ||
| 8 | | 8 | ||
| 7.3 and contents of MantidConfig folders | | 7.3 and contents of MantidConfig folders | ||
− | | | + | | [https://github.com/mantidproject/mantid/blob/master/scripts/Inelastic/Direct/ISISDirecInelasticConfig.py Mantid configuration script], started by [https://github.com/FITLinuxSupport/analysisscripts/blob/master/Build_Mantid_config.py issicompute configuration script] |
| | | | ||
− | + | # Mantid configured for first usage, [[Experimental Data]] and [[Instrument parameters and Instrument definitions]] data search path is set up and path to common inelastic script is configured. | |
− | + | # User scripts copied to users according their instruments and the contents of [https://github.com/mantidproject/scriptrepository/tree/master/direct_inelastic Mantid Users Script repository] | |
− | |||
− | |||
− | |||
− | |||
− | |||
| Alex Buts | | Alex Buts | ||
|- | |- | ||
Line 89: | Line 92: | ||
| Once an Hour: [https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Inelastic data update script] pulls contents of the web repositories into files on isiscompute. | | Once an Hour: [https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Inelastic data update script] pulls contents of the web repositories into files on isiscompute. | ||
| [[Instrument parameters and Instrument definitions]] files in correspondent isiscompute folders are up-to date | | [[Instrument parameters and Instrument definitions]] files in correspondent isiscompute folders are up-to date | ||
− | | Jeff Warren | + | | Jeff Warren & Alex Buts |
|} | |} | ||
− | + | [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid] | |
− | |||
=== Extract from .json file defined users who has access to the data === | === Extract from .json file defined users who has access to the data === | ||
Line 123: | Line 125: | ||
{ "NEXT RB NUMBER"" | { "NEXT RB NUMBER"" | ||
} | } | ||
+ | |||
+ | |||
+ | [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid] |
Latest revision as of 17:12, 9 January 2023
On 01/01/2023 this information become outdated. IDAaaS Support Analysis team is solely responsible for auto-configuring IDAaaS machines. The information below is remaining for references purposes and stating how it used to work in the past.
Historical automatic user configuration procedure
When an experiment is approved and scheduled by instrument scientist, User office assigns a user with his/her federal ID, which allows user to access ISIS network including IDAaaS virtual machines and provides appropriate information to [Catalog service], giving the user rights to access the data through the Catalog.
At some stage, special information file, which defines the participants of the experiments and the scientists who have access to the data of this experiment is placed on the internal server where it is accessed and automatically processed by issicompute configuration script. The format of the file is .json and the sample contents is provided below.
The automatic isiscompute configuration script runs twice a day on isiscompue server processing the information file, creating user accounts and appropriate RB folders and giving users permissions to access their RB folders on the basis of the information file. Alex Buts and FBU IT support FBU IT support (Jeffs Warren) have access and can modify this script. At the final stage this script runs the Mantid configuration script for inelastic instruments, which configures each user's Mantid to use appropriate Experimental Data, Instrument parameters and Instrument definitions, necessary to reduce experimental data. It also provides users with initial version of Reduction scripts, which users should modify to their requests according to the description, using help of their instrument scientist if necessary. Alex Buts is responsible for maintaining this script.
When initial Reduction scripts are copied to user's RB folder from the Mantid Script Repository sample folder, script date is set to the experiment start date and the time -- to the exact midnight of the start date. Configuration script running twice a day checks this date and if it does not changed, replaces the existing script with the script, found in the Mantid Script Repository sample folder. If somebody touched the copied script, its date and time changes and autoconfiguration script stops replacing the file.
The diagram and the table below summarize the user creation workflow:
ISISOMPUTE User creation workflow
Less visual but more detailed description of the workflow is provided in the table below. The number of the row in the table corresponds to the number of the block in the diagram above.
Stage Num | Input | Frequency, Actor & Event | Result | Person responsible |
---|---|---|---|---|
1 | -- | Two weeks before Cycle, Crhis closes archive on the previous cycle. Sets up new cycle | Cycle Folders appear in the archive. | Chris Moreton-Smith |
2 | --- | Somebody from the team sets up new cycle for cataloguing data. | Data are searchable through iCat. | Tom Griffin |
3 | Results of the review pannel | On request, Instrument Scientists, schedules experiment in Scheduler application | Input for scheduler script | Instrument Scientist |
4 | e-mail to PI from 5.1, about experiment accepted | Experimental team is added to visits system | Input for scheduler scripts | PI on accepted proposal |
5 | Results from 3 & 4 | Twice a day: Scheduler scripts runs processing inputs. |
|
Contact responsible from data acquisition support group |
6 | Output 5.1 from Scheduler script | Create FedID & renew the old one |
|
User's office |
7 | Output 5.2 from Scheduler script | Twice a day: issicompute configuration script |
|
Jeff Warren & Alex Buts |
8 | 7.3 and contents of MantidConfig folders | Mantid configuration script, started by issicompute configuration script |
|
Alex Buts |
9. | Contents of Mantid Users Script repository on the web. Contents of SVN servers on the web. | Once an Hour: Inelastic data update script pulls contents of the web repositories into files on isiscompute. | Instrument parameters and Instrument definitions files in correspondent isiscompute folders are up-to date | Jeff Warren & Alex Buts |
Extract from .json file defined users who has access to the data
{ "Last Updated": 20150413, "experiments": [ { "RbNumber": "1234567", "Instrument": "MERLIN", "Cycle": "2013/4", "StartDate": "20131203", "Permissions": [ { "email": "YYYY@stfc.ac.uk", "fedid": "XXXXX" }, { "email": "warren.jeffs@stfc.ac.uk", "fedid": "gpq43739" }, { "email": "alex.buts@stfc.ac.uk", "fedid": "wkc26243" } ] }, { "NEXT RB NUMBER"" }