Reduction scripts - Revision history

Alex Buts: /* Reduction scripts used to process the experimental results */

2023-01-09T16:22:08Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2023-01-09T16:20:42Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2023-01-09T16:19:24Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2023-01-09T16:16:56Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2017-09-13T14:52:50Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2017-05-02T08:54:29Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2016-06-22T12:05:50Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2016-04-26T19:14:22Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2016-04-26T18:36:26Z

Reduction scripts used to process the experimental results

Alex Buts: /* Reduction scripts used to process the experimental results */

2016-04-26T18:34:42Z

Reduction scripts used to process the experimental results

← Older revision		Revision as of 16:22, 9 January 2023
Line 33:		Line 33:

	As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.		As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.
		+
		+	[http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid]

← Older revision		Revision as of 16:20, 9 January 2023
Line 32:		Line 32:
	[https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Small configuration script] runs on IDAaaS every hour, and together with updating [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument Parameters repository], updates git repository, so all files changed or added by instrument scientists become available on all IDAaaS VM or local instrument machines within an hour. Members of excitation staff group can update the repository on request. See [[Automatic user configuration]] pages for detailed description of things, done to provide users with up-to-date data and parameters.		[https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Small configuration script] runs on IDAaaS every hour, and together with updating [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument Parameters repository], updates git repository, so all files changed or added by instrument scientists become available on all IDAaaS VM or local instrument machines within an hour. Members of excitation staff group can update the repository on request. See [[Automatic user configuration]] pages for detailed description of things, done to provide users with up-to-date data and parameters.

−	As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.	+	As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.
−
−	<span style="color:#0000ff; background:#FF69B4"> [http://www.isis.stfc.ac.uk/instruments/mari/mari4765.html MARI] instrument scientists who use small instrument parameter files, were able to use '''Mantid script repository''' for instrument files, but even this does not work very well. Any other instrument just can not push sufficient size files to Git.
−	~~</span>~~
−
−	~~<span style="color:#0000ff; background:#FF69B4">~~
−	At the fall of 2015 GitHub introduced [https://git-lfs.github.com/ Large file storage] so instrument files can be now stored in GitHub, though Mantid should be modified to use such possibility </span>
−
−
−	~~[http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid]~~

@@ Line 30: / Line 30: @@
 To have write access to the repository through Mantid (and see what is actually there using Mantid GUI) users have to install '''Mantid script repository''' through Mantid (or Git) into their accounts.
-[https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Small configuration script] runs on isiscompute every hour, and together with updating [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument Parameters repository], updates git repository, so all files changed or added by instrument scientists become available on isiscompute within an hour.  This script operations are logged to:
+[https://github.com/FITLinuxSupport/analysisscripts/blob/master/UpdateSVN Small configuration script] runs on IDAaaS every hour, and together with updating [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument Parameters repository], updates git repository, so all files changed or added by instrument scientists become available on all IDAaaS VM or local instrument machines within an hour.  Members of excitation staff group can update the repository on request. See [[Automatic user configuration]] pages for detailed description of things, done to provide users with up-to-date data and parameters.
 As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.

@@ Line 17: / Line 17: @@
 Physically the '''Mantid script repository''' is located on [https://github.com/mantidproject/scriptrepository GitHub] and is normal Git repository, which is accessed through simplified Mantid GUI interface, but can also be operated by a person who has github account and is familiar with Git by accessing [https://github.com/mantidproject/scriptrepository GitScriptRepository] link and performing normal git operations.
-[https://isiscompute.nd.rl.ac.uk/nxwebplayer Isiscompue servers] have Git repository cloned at
+[https://isis.analysis.stfc.ac.uk/workspaces/ IDAaaS workspaces] have Git repository cloned at
    /opt/UserScripts/

@@ Line 1: / Line 1: @@
 == Reduction scripts used to process the experimental results ==
-ISIS reduction scripts contain number of interconnected scripts which are the part of Mantid and can be located in ''Mantid Installation Folder/scripts/Inelastic/Direct/'' or see [https://github.com/mantidproject/mantid/tree/master/scripts/Inelastic/Direct Mantid distribution on the web]. These scripts set up and run various [http://docs.mantidproject.org/nightly/algorithms/index.html Mantid algorithms] to process input data according to the instrument related settings defined in [https://github.com/mantidproject/mantid/tree/master/instrument Instrument parameters files] and  the settings, specific to a particular experiment. The later settings have to be done by instrument scientist and user locally in special scripts, allowing user to make these settings and run Mantid to reduce their data. An instrument specific examples of such scripts are part of Mantid continuous integration system and can be accessed on the web: (see, e.g. [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MariReduction.py MARI], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MAPS_DGSReduction.py MAPS], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MERLINReduction.py MERLIN] and [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_LETReduction.py LET] or [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/ISIS_direct_reduction_script_description their separate description] within this wiki)
+ISIS reduction scripts contain number of interconnected scripts which are the part of Mantid and can be located in ''Mantid Installation Folder/scripts/Inelastic/Direct/'' or see [https://github.com/mantidproject/mantid/tree/master/scripts/Inelastic/Direct Mantid distribution on the web]. These scripts set up and run various [http://docs.mantidproject.org/nightly/algorithms/index.html Mantid algorithms] to process input data according to the instrument related settings defined in [https://github.com/mantidproject/mantid/tree/master/instrument Instrument parameters files] and  the settings, specific to a particular experiment. The later settings have to be done by instrument scientist and user locally in special scripts, allowing user to make these settings and run Mantid to reduce their data. An instrument specific examples of such scripts are part of Mantid continuous integration system and can be accessed on the web: (see, e.g. [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MariReduction.py MARI], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MAPS_DGSReduction.py MAPS], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MERLINReduction.py MERLIN] and [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_LETReduction.py LET] or [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/ISIS_direct_reduction_script_description. their separate description] within this wiki)
 Instrument scientists modify these scripts according to the requests of a particular experiment and Mantid provides special way to share these scripts through '''Mantid script repository''' configured at

@@ Line 34: / Line 34: @@
   /var/log/UpdateSVN.log.
-which is available on '''ndmlnx01''' node only. Root can update the repository on request. See [[Automatic user configuration]] pages for detailed description of things, done to provide users with up-to-data data and parameters.
+which is available on '''ndmlnx01''' node only. Root can update the repository on request. See [[Automatic user configuration]] pages for detailed description of things, done to provide users with up-to-date data and parameters.
 As '''Mantid script repository''' is a git repository, it has to obey the constrains imposed by GitHub. One of this constrains is the limitation on the size of the files allowed to be stored on GitHub. This does not allow to use Git repository for storing [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data Instrument files], necessary for data analysis, so local svn repository, described in [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data another chapter], has to be used instead.

@@ Line 3: / Line 3: @@
 ISIS reduction scripts contain number of interconnected scripts which are the part of Mantid and can be located in ''Mantid Installation Folder/scripts/Inelastic/Direct/'' or see [https://github.com/mantidproject/mantid/tree/master/scripts/Inelastic/Direct Mantid distribution on the web]. These scripts set up and run various [http://docs.mantidproject.org/nightly/algorithms/index.html Mantid algorithms] to process input data according to the instrument related settings defined in [https://github.com/mantidproject/mantid/tree/master/instrument Instrument parameters files] and  the settings, specific to a particular experiment. The later settings have to be done by instrument scientist and user locally in special scripts, allowing user to make these settings and run Mantid to reduce their data. An instrument specific examples of such scripts are part of Mantid continuous integration system and can be accessed on the web: (see, e.g. [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MariReduction.py MARI], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MAPS_DGSReduction.py MAPS], [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_MERLINReduction.py MERLIN] and [https://github.com/mantidproject/mantid/blob/master/Testing/SystemTests/tests/analysis/ISIS_LETReduction.py LET] or [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/ISIS_direct_reduction_script_description their separate description] within this wiki)
-Instrument scientists modify these scripts according to the requests of particular experiment and Mantid provides special way to share these scripts through '''Mantid script repository''' configured at
+Instrument scientists modify these scripts according to the requests of a particular experiment and Mantid provides special way to share these scripts through '''Mantid script repository''' configured at
    File->Script Repository

← Older revision		Revision as of 19:14, 26 April 2016
Line 43:		Line 43:
	<span style="color:#0000ff; background:#FF69B4">		<span style="color:#0000ff; background:#FF69B4">
	At the fall of 2015 GitHub introduced [https://git-lfs.github.com/ Large file storage] so instrument files can be now stored in GitHub, though Mantid should be modified to use such possibility </span>		At the fall of 2015 GitHub introduced [https://git-lfs.github.com/ Large file storage] so instrument files can be now stored in GitHub, though Mantid should be modified to use such possibility </span>
		+
		+
		+	[http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Setting_up_Mantid,_Matlab_and_auxiliary_programs_to_process_results_of_inelastic_experiments. Back to setting up Mantid]

@@ Line 12: / Line 12: @@
 The repository, installed locally according to the wizard invoked from this menu allows to download scripts located in the repository and share
-the changes made to these files with other users of this repository.  In addition to data exchange, ScriptRepository wizard configures Python path, to make repository's scripts runnable from Mantid. When Mantid starts, it updates its  local copy of the '''Mantid script repository''' from the web. The progress of this operations reported to Mantid log window.
+the changes made to these files with other users of this repository.  In addition to data exchange, ScriptRepository wizard configures Python path, to make repository's scripts runnable from Mantid. When Mantid starts, it updates its  local copy of the '''Mantid script repository''' from the web. The progress of this operations reported to the Mantid log window (shown as the background of the fig above)
 Default parameters of these scripts are part of Mantid and are described [http://shadow.nd.rl.ac.uk/wiki/idr/index.php/Location_and_details_of_access_to_instrument_parameters,_necessary_to_reduce_data#Instrument_Definition_Files in Instrument Definition Files] section of this Wiki.

@@ Line 8: / Line 8: @@
 field of Mantid GUI (see field pointed at as 1 on the picture below):
 [[File:InstallScriptRepositorye.png]]
 The repository, installed locally according to the wizard invoked from this menu allows to download scripts located in the repository and share
 the changes made to these files with other users of this repository.  In addition to data exchange, ScriptRepository wizard configures Python path, to make repository's scripts runnable from Mantid. When Mantid starts, it updates its  local copy of the '''Mantid script repository''' from the web. The progress of this operations reported to Mantid log window.