MAJA Installation

MAJA Overview

MAJA (for Maccs-Atcor Joint Algorithm), is an atmospheric correction and cloud screening software based on the MACCS processor, developped for CNES by CS GROUP - France, from a method and a prototype developped at CESBIO, 1 2 3. In 2017, thanks to an agreement between CNES and DLR and to some funding from ESA, we started adding methods from DLR 's atmospheric correction software ATCOR into MACCS. MACCS then became MAJA. Currently, MAJA allows the following processing steps: Atmospheric correction Cloud detection Estimation of the Aerosol-Optical-Depth (AOT) Correction of environmental- and slope-effects MAJA is based on a multi-temporal method - allowing to refine the outputs using the previous inputs. Check the chapters 'Running MAJA' and 'Generating the documentation' in order to get more information about this.

MAJA source page

MAJA is only available on linux OS.

MAJA is an external tool that can be used directly in the command line, but some of its application has been adapted in order to be able to execute them from SNAP. For using from SNAP, it is needed to install the "Optical Toolbox MAJA Adapters" plugin which establishes a "bridge" between SNAP and MAJA. As the plugin is a "bridge", it is needed to install MAJA. The easiest way to install it is directly from SNAP.

MAJA installation from SNAP

For now the installer have to install from local MAJA installer. Please, download MAJA-4.x.x installer in the following link:


MAJA source page

• Install the MAJA plugin in SNAP: Tools->Plugins

• Check the installation in the menu: Optical>Thematic Land Processing>Maja
• Install MAJA from SNAP
• Tool->Manage External Tools
• Select one of the MAJA applications and Edit

• Download MAJA installer 4.x.x, specify the installer path and click on "Install Now".

• Check that it has been installed properly

Custom MAJA Installation

• Download precompiled version of the MAJA (only version 4.x.x is supported) The following instructions will set you up to get a working copy on your system.
  Maja is available as a self-extracting archive via the following link: MAJA download page
• You just have to unzip the provided package and use the following command :
./MAJA-4-x.run --target /path/to/install
• Tool->Manage External Tools
• Select one of the MAJA applications and Edit

• Go to System Variables and edit MAJA_BIN_DIR variable with your custom installation

Running Maja

• The MAJA SNAP plugin use startmaja supervisor. It enables to process successively all files in a time series of Sentinel-2 images for a given tile, stored in a folder. The initialisation of the time series is performed with the "backward mode", and then all the dates are processed in "nominal" mode. The backward mode takes much more time than the nominal mode. On my computer, which is a fast one, the nominal mode takes 15 minutes, and the backward mode takes almost one hour. No control is done on the outputs, and it does not check if the time elapsed between two successive products used as input is not too long and would require restarting the initialisation in backward mode.
  
  The description below will explain how to process a set of data above tile 31TFJ, near Avignon in Provence, France. To process any other tile, you will need to prepare the DTM and store the data in the DTM folder.

Prepare folders and input files

• To use the start_maja script, you need to configure the directories, within the folder.txt file.
  Here is an example of configuration that must be adapted to your own directory structure.
  [Maja_Inputs]
repWork=./work
repGipp=./gipp
repMNT=./dtm
repL1 =/path/to/L1C
repL2 =/path/to/L2A
exeMaja=/path/to/bin/maja
repCAMS=/path/to/CAMS

[DTM_Creation]
repRAW=./dtm/raw
repGSW=./dtm/gsw

• repWork is a directory to store the temporary files
• repGipp is the folder where Start_maja automatically downloads the GIPP-set for each plugin.
• repMNT stores the DTM (MNT in french) in Maja format
• repL1 is where to find the L1C data (without the site name which is added aferward optionally)
• repL2 is for the L2A data (without the site name which is added aferwards, optionally again)
• exeMaja is where the Maja binary exe is located. In case of the MAJA-SNAP installation, the binary is located in $HOME/.snap/auxdata/MAJA-4.2.1/bin/
• repCAMS is where CAMS data is stored. You do not need to specify this directory if you decide to not process with CAMS option.
• repRAW stores the raw DTM archives (such as the ones for SRTM, which have the name `srtm_37_04.zip`)
• repGSW stores the raw Water-Mask files (such as the one for GSW, which have the name `occurrence_0E_50N*.tif`)


To run MAJA, Start_maja copies all the necessary data in a temporary input folder. Here is an example of its content in nominal mode.

Click to expand folder structure.

S2A_MSIL1C_20200313T095031_N0209_R079_T33UYQ_20200313T102505.SAFE
S2A_TEST_GIP_CKEXTL_S_ALLSITES_00001_20190626_21000101.EEF
S2A_TEST_GIP_CKEXTL_S_ALLSITES_10001_20190626_21000101.EEF
S2A_TEST_GIP_CKQLTL_S_ALLSITES_00001_20190626_21000101.EEF
S2A_TEST_GIP_CKQLTL_S_ALLSITES_10001_20190626_21000101.EEF
S2A_TEST_GIP_L2ALBD_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2A_TEST_GIP_L2ALBD_L_CONTINEN_00001_20190626_21000101.HDR
S2A_TEST_GIP_L2COMM_L_ALLSITES_00001_20190626_21000101.EEF
S2A_TEST_GIP_L2COMM_L_ALLSITES_10001_20190626_21000101.EEF
S2A_TEST_GIP_L2DIFT_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2A_TEST_GIP_L2DIFT_L_CONTINEN_00001_20190626_21000101.HDR
S2A_TEST_GIP_L2DIRT_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2A_TEST_GIP_L2DIRT_L_CONTINEN_00001_20190626_21000101.HDR
S2A_TEST_GIP_L2SMAC_L_ALLSITES_00001_20190626_21000101.EEF
S2A_TEST_GIP_L2TOCR_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2A_TEST_GIP_L2TOCR_L_CONTINEN_00001_20190626_21000101.HDR
S2A_TEST_GIP_L2WATV_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2A_TEST_GIP_L2WATV_L_CONTINEN_00001_20190626_21000101.HDR
S2B_TEST_GIP_CKEXTL_S_ALLSITES_00001_20190626_21000101.EEF
S2B_TEST_GIP_CKEXTL_S_ALLSITES_10001_20190626_21000101.EEF
S2B_TEST_GIP_CKQLTL_S_ALLSITES_00001_20190626_21000101.EEF
S2B_TEST_GIP_CKQLTL_S_ALLSITES_10001_20190626_21000101.EEF
S2B_TEST_GIP_L2ALBD_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2B_TEST_GIP_L2ALBD_L_CONTINEN_00001_20190626_21000101.HDR
S2B_TEST_GIP_L2COMM_L_ALLSITES_00001_20190626_21000101.EEF
S2B_TEST_GIP_L2COMM_L_ALLSITES_10001_20190626_21000101.EEF
S2B_TEST_GIP_L2DIFT_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2B_TEST_GIP_L2DIFT_L_CONTINEN_00001_20190626_21000101.HDR
S2B_TEST_GIP_L2DIRT_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2B_TEST_GIP_L2DIRT_L_CONTINEN_00001_20190626_21000101.HDR
S2B_TEST_GIP_L2SMAC_L_ALLSITES_00001_20190626_21000101.EEF
S2B_TEST_GIP_L2TOCR_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2B_TEST_GIP_L2TOCR_L_CONTINEN_00001_20190626_21000101.HDR
S2B_TEST_GIP_L2WATV_L_CONTINEN_00001_20190626_21000101.DBL.DIR
S2B_TEST_GIP_L2WATV_L_CONTINEN_00001_20190626_21000101.HDR
S2__TEST_AUX_REFDE2_33UYQ_0001.DBL.DIR
S2__TEST_AUX_REFDE2_33UYQ_0001.HDR
S2__TEST_GIP_L2SITE_S_ALLSITES_00001_20190626_21000101.EEF
SENTINEL2B_20200308-095659-128_L2A_T33UYQ_C_V1-0


• .SAFE file is the input L1C product
• `SENTINEL2_*_L2A_*` files are the L2A products, which is the result from a previous run of MAJA Then for each of the following items you will have a `DIR` or `DBL` folder/archive and a `HDR` file:
• GIP are parameter files for each satellite (S2A or S2B in this example)
• REFDE2 files are the DTM (Digital Terrain Model) files. How to obtain them is explained in `prepare_dtm`.



A "userconf" folder is also necessary, but it is already provided by Start_Maja.

Retrieve Sentinel-2 L1C data.

The use of peps_download.py to download Sentinel-2 L1C products is recommended : https://github.com/olivierhagolle/peps_download



• For instance, with peps_download.py (you need to have registered at https://peps.cnes.fr and store the account and password in peps.txt file. `python ./peps_download.py -c S2ST -l 'Avignon' -a peps.txt -d 2017-01-01 -f 2017-04-01 -w /path/to/L1C_DATA/Avignon`
  
  
• Some users tend to store the data per site. A given site can contain several S2-tiles. All the L1C tiles corresponding to a site are stored in a directory named /path/to/L1C_DATA/Site
  

Create DTM

A DTM folder is needed to process data with MAJA which needs to have the same geographical extent as the L1C input product - It depends on the tile you want to process. A tool exists to create this DTM, [it is available in the "prepare_mnt" folder](StartMaja/prepare_mnt/Readme.md). When using Start_maja the creation of the folder in Maja-format is automatically attempted. For this the `repRAW` and `repGSW` directories in your `folders.txt` file need to be set.

Download CAMS data

If you intend to use the data from Copernicus Atmosphere Monitoring Service (CAMS), that we use to get an information on the aerosol type, you will need to download the CAMS data.

CAMS data can be downloaded after a simple registration, but these days, probably due to a large success, it takes more than a day to download a day of CAMS data. Through agreements with ECMWF or through your countries weather agency, it is possible to get a priviledged access, which grants far better performances. To get a better access, it is also possible to download data month per month, instead of day per day. Donwloading a month takes almost the same time as downloading a day. But it does not work for real time processing, which needs day per day downloads.

If you want to use CAMS option, follow [cams_download tool instructions](StartMaja/cams_download)

GIPP Files

MAJA uses GIPP-files (Ground Image Processing Parameters) to configure the different algorithms of the chain without having to recompile the code. You can find all sets here :

http://osr-cesbio.ups-tlse.fr/gitlab_cesbio/kettigp/maja-gipp

Start_Maja automatically downloads the GIPPs necessary for each plugin and links them to the folder. You only need to modify the parameters if you want to change the behavior of the processing chain.

userconf

On top of the GIPP files, a global user configuration folder is used, named `userconf`. Should you want to debug the processing chain, we recommend to check the parameters that are listed in the `MAJAUserConfigSystem.xml` or for each plugin `MAJAUserConfig_*xml`.

Known Errors

Some Sentinel-2 L1C products lack the angle information which is required by MAJA. In this case, MAJA stops processing with an error message. This causes issues particularly in the backward mode. These products were acquired in February and March 2016 and have not been reprocessed by ESA (despite repeated requests). You should remove them from the folder which contains the list of L1C products to process.