DIRAC v7r0

DIRACOS

As of v7r0, Externals and Lcgbundle packages are replaced by DIRACOS. Note that runit, as well as mySQL, are not anymore distributed, and you must install them by yourself. RPMs are provided here: http://diracproject.web.cern.ch/diracproject/rpm/

Please follow migration instructions here

Pilot 3

The "Pilot 3" becomes the default Pilot version. Technically, it means that:

• the Pilot code will now be taken from https://github.com/DIRACGrid/Pilot (the location of "Pilot2 code" is instead https://github.com/DIRACGrid/DIRAC/tree/rel-v6r22/WorkloadManagementSystem/PilotAgent)
• if you have an extension of the Pilot2 code, you will need to move it to a new git repository, normally called "[YourCommunity]Pilot". For an example, please look at https://gitlab.cern.ch/lhcb-dirac/LHCbPilot
  • in case you don't use git for hosting your "[YourCommunity]Pilot", a development would be needed. Please get in touch.
  • in case you use git, but the repository is not public, also get in touch.
• for more info please refer to https://dirac.readthedocs.io/en/rel-v7r0/AdministratorGuide/Systems/WorkloadManagement/Pilots/index.html
• if you really want to keep using the Pilot2 code (and maybe its extension) you can keep doing it by setting the SiteDirector flag "Pilot3 = False" (https://dirac.readthedocs.io/en/rel-v7r0/AdministratorGuide/Systems/WorkloadManagement/Pilots/Pilots3.html#starting-pilot3-via-sitedirectors)
  • The pilot2 code will be removed in a future DIRAC release, and it's, from now on, legacy unsupported code.

Core

[TECH PREVIEW] It is now possible to use M2Crypto instead of PyGSI by setting the environment variable DIRAC_USE_M2CRYPTO=Yes in the bashrc. This is a technology preview as described here https://dirac.readthedocs.io/en/integration/AdministratorGuide/technologyPreviews.html

AccountingSystem

PR https://github.com/DIRACGrid/DIRAC/pull/3701 merges the ObjectLoaders and TypeLoaders used in AccountingSystem and MonitoringSystem into a single one. Pay attention in case you use them.

ALTER the AccountingDB tables that use "MinorStatus", so that it is a `VARCHAR(256). Some examples follow (adapt them to your installation):

ALTER TABLE `ac_key_DIRAC-Certification_Job_FinalMinorStatus` MODIFY value VARCHAR(256) NOT NULL;

Also consider increasing the size of "Site" and "User" fields to 64: https://github.com/DIRACGrid/DIRAC/pull/4290

WorkloadManagementSystem

• The service JobStateSyncHandler doesn't exist anymore, please remove it case you have it installed.
• New service PilotManager is introduced to manipulate pilot related information. It should be added to installations migrating to v7r0.

Jobs parameters on ElasticSearch backend

• [TECH PREVIEW] It's possible (and advised) to store the Jobs parameters on ElasticSearch (up to now, they have been stored in a key/value pair MySQL table, part of JobDB). This is a technology preview as described here https://dirac.readthedocs.io/en/integration/AdministratorGuide/technologyPreviews.html

TransformationSystem

PR https://github.com/DIRACGrid/DIRAC/pull/3703 replaces the TransformationInputDataQuery table with a new table TransformationMetaQueries, which stores both Input and Output MetaQueries. In order to guarantee backward compatibility, before deploying the release, it's necessary to rename the TransformationInputDataQuery table:

  ALTER TABLE `TransformationInputDataQuery` RENAME `TransformationMetaQueries`, 
  ADD COLUMN QueryType VARCHAR(8), 
  CHANGE COLUMN `ParameterName` `MetaDataName` VARCHAR(512) NOT NULL, 
  CHANGE COLUMN `ParameterValue` `MetaDataValue` blob NOT NULL,  
  CHANGE COLUMN `ParameterType` `MetaDataType` VARCHAR(8) NOT NULL;

And create a view as following:

  CREATE OR REPLACE VIEW `TransformationInputDataQuery` AS 
  SELECT TransformationID, 
  MetaDataName as ParameterName, 
  MetaDataValue as ParameterValue,  
  MetaDataType as ParameterType
  from TransformationMetaQueries;

Transformation: Configuration Option Change

• In case you used in the past the option found in the /Operations section of your Configuration called "Productions/ProductionFilesMaxResetCounter": it was renamed to "Transformations/FilesMaxResetCounter".

ProductionSystem

With v7r0, with PR https://github.com/DIRACGrid/DIRAC/pull/3703, a completely new system is put in place: the ProductionSystem, which is used for high-level productions management.

Install the DB with:

CREATE DATABASE ProductionDB;

To install the Production System service from the admin console:

install service Production ProductionManager

DataManagementSystem

Removed old FTS system

The old FTS system has been completely removed.

Removed SRM2Storage and XROOTStorage plugins

These plugins are superseeded for a long time by GFAL2_SRM2Storage and GFAL2_XROOTStorage

** MonitoringSystem**

Introduction of a configuration parameter IndexPrefix in the configuration system under:

Systems/Monitoring/<Setup>/Databases/MonitoringDB/IndexPrefix

If this parameter is present in the CS, all the indices that are created on the Elasticsearch instance are prefixed with this string. If this parameter is not set, the setup name is used to prefix indices (default before this change).