4SM flowchart

Your comments are invited See you on 4SM blog
Where is the competition?

4SM procedure 2017

see 4SM demonstration
see also HYPHOON

**Procedure for operating 4SM.7.00**
this file is displayed each time you run -MakePIX Raw image data============================= may be formated in a variety of formats 4SM uses PCIDSK format for its working database.pix TIF format is OK, but must first be exported to PCIDSK OpenEV uses GDAL for handling very many image formats Seatruth data================================== DTM: best is a gridded digital terrain model overlay, co-registered XYZ: depth sounding profiles as ASCII text file are also OK Georeferencing================================== UTM please Metadata================================== for OLI data, textfile L8..._MTL.txt must be available for WV2 data, textfile ..........IMD must be available I use OpenEV to export raw data to PCIDSK image format specify a subwindow if required resample the PAN band as required Format the command line script as required use an existing command line update/modify as required pay special attention to georeferencing details and byte_swap Run -MakePIX=================================== to write database header to write database.pix structure, FILE interleaving and stop Run -Import...================================= to import raw 16U data into database and stop Run -Prepare_Scale============================== to write 8 bits scaled from 16 bits raw data and stop Display 8 bits scaled image in OpenEV beware of byte_swap investigate using various enhancement schemes get a feeling of the image radiometric quality get a feeling of illumination conditions get a feeling of atmospheric conditions get a feeling of homogeneity of water bodies look for negative bottom areas which areas of this image are worthy of shallow water work? Prepare shapefiles: Lsw.shp and dLsw.shp============ Lsw should be as low as possible: deep water radiance cloud shadows or the area with lowest radiance in the NIR/SWIR band dLsw.shp to locate deep water radiance at an optimized location Prepare shapefiles: glint.shp==================== must absolutely sample optically deep areas in the blue/green range to the extent possible, glint.shp should avoid clouds and whitecaps polygons and PolyLines may be mixed Prepare shapefiles: LsM.shp and mSOIL.shp======= LsM.shp should sample bright genuine dryland LsM is one point to represent maximum brightness of shallow bottoms either with Points or Polygons/PolyLines. Beware of wetlands Prepare shapefiles: mSOIL.shp==================== mSOIL.shp may include less bright dryland. Beware of wetlands mSOIL shall give a sense of the Soil Line through a scatter of select pixels either with Points or Polygons/PolyLines. Beware of wetlands Prepare mask_1.shp shapefile================== these unsafe marine areas shall be skipped while selecting calibration data like polluted areas, wave breakers, marinas, ships, alien floating things, etc Prepare land.shp shapefile================== these areas shall be forced to land Prepare vegetation.shp shapefile================== this shall help estimate water volume reflectance in the Blue range Prepare shallow.shp shapefile================== these areas shall be forced to marine Run AutoCalibrator: does the following:======== reads raw radiances from 16 bits raw data channels in database.pix scales raw radiances from 16 bits range to 8 bits range scans the image for nodata pixels extracts and format the glint_regressions.gli textfile formats glint_regressions.sh script Lsw: computes provisional Lsw deep water radiance Lw : computes provisional Lw water volume reflectance La : computes provisional La path radiance prepares the special effect mask BPL and DPL: extracts the calibration data for BPL and DPL BDH: extracts the calibration data for bidimensional histograms runs the AutoCalibrator displays the provisional AutoCalibration diagram formats and displays the 4SM image-specific command line and stops this provisional calibration diagram may now be inspected Are you satisfied with the mask mSE===== if not, see what needs to be modified, and start again Run glint_regressions.sh======================= this displays the various glint regressions, pair wise investigate the glint regressions if not satisfied, modify the glint.shp shapefile and start again Display the glint text file==================== investigate the glint_regressions_text_file.gli Run ABProfiler================================= in order to investigate deglinting performance, and Lsw and Lw -ProfileAB/LVW/profile_red/chAB_6_7 see whether dLsw should be specified to adjust Lsw adjust as required, until satisfied ensure Lw is consistent with the physics of underwater light field ensure La is consistent with the physics of atmospheric light field Deglint the image============================== -Deglint/vRbad/q/GlintM20.0 and adjust as required until satisfied Run Extractor================= if mot satisfied with provisional calibration dataset specify your conditions for BPL: deglinted or not specify your conditions for BPL: mask mSE condition specify your conditions for BPL: NIRmax threshold specify your conditions for BDH: deglinted or not specify your conditions for BDH: whole image or marine only specify your conditions for extraction: whole image or select window -Extract/v/rawBDH/fullBDH/NIRband7/NIRmax5/mBPL2/mask_3 Run Calibrator================= then only, engage in the calibration process -Calibrate/V/BDSNpzg/plot_BPL/BC_6_5_3_2/BDh_12_13_34 establish/adjust LsM establish Ki/Kj for all pairs, as precisely as possible estimate Lw for all bands: very sensitive for dark botoms consolidate your choices for LsM, Lsw and Lw Run Modeler then only you may start modeling deglinting range may be tuned smoothing parameters may be tuned water column correction may be specified TOA or BOA Bmin is specified in order to alleviate very dark bottom artefacts mask_deep may be used to mask out deep water areas and various other operational tricks Run ABProfiler to investigate your results -ProfileAB/ZrvwB/profile_blue/dTM/chAB_1_2_3 adjust command line as required until satisfied Start again until satisfied Run ZZRegressor for sea truth evaluation Et voila! you are now ready for bottom typing? 4SM_LUT use a LUT to map the spatial variations of the effective diffuse attenuation coefficient When finished: run the -Archiver
It takes me ~5 hours to process a new Landsat 8 OLIP scene, from extracting the image downloaded from USGS, to running seatruth tests and reporting to the web

more details below
WARNING:
Due to progress achieved since january 2009,
many aspects of the HELP and HOWTO services are outdated

Use OpenEV to prepare your raw image

Get hold of ancillary infos:
- nbRow, nbLin, nb of BGRNir channels
- sensorType and sensorMode
- georeferencing infos
- pixel size
- Export your raw image from your petty format to PCIDSK format
- imaging conditions
- wavelengths...

Use 4SM to create a PCIDSK formated database

Specify the -DB... -MIS... -MakePIX... commandline arguments and run your script

If 16 bits data, use OpenEV to sample some bright bareland
- this will be used by 4SM to scale the data from 16 bits to 8 bits (floating)

For 16 bits data, a -SCL... commndline argument must be specified

Use 4SM to import your raw data channels
- HowTo Import

Specify the -Import... commandline argument and run your script

Use OpenEV to create some shapefiles

Do some optional digitizing for house keeping
- NoData.shp
- baddata.shp
- land.shp
- mSOIL.shp for bright bare soil
- shallow.shp
- badwaters.shp
- clouds.shp
- waves.shp
- deep.shp for optically deep water
Do some compulsory digitizing of training sites
- Lsw.shp for deep water radiance
- LsM.shp for bright bare soil
- glint.shp for glint/haze regression
- vegetation.shp for healthy vegtation

Use 4SM to create and fine-tune a maskSE
- HowTo Make MaskSE

Add a -AutoCAL... argumentand run your script
- this shall create the database structure with all required channel
- then run a few statistics
- then work out the glint regressions and write a database.gli textfile, and display the regressions
- then automtically write the special effect mask mSE
- then extract the deglinted calibration data textfile
- then attempt to run a calibration
- then offer a fully developped commandline script

If all gos well, 4SM offers
a fully developped commandline script

which now needs to be verified
before use

Use 4SM to deglint your data
- HowTo Deglint

Add a -Deglint... commandline argument
and run your script: this shall

work out a glint regression
then write a deglinted image

Use 4SM to extract a calibration data textfile
- How to calibration textfile

disable -deglint... commandline argument
Add an -Extract... commandline argument
and run your script
this shal write a datbase.cal calibration textfile

Use 4SM to calibrate the simplified shallow water radiative transfer optical model, using the calibration data text file
- HowTo Calibrate
- HowTo Calibration Diagram

Add the following commandline arguments
- -KK...
- -Z...
- -B...
- -Calibrate...
and run your script
optional arguments are
- -deglint... (recommanded)
- -Smooth... (recommanded)

Use 4SM to operate the simplified shallow water radiative transfer optical model on all shallow water pixels

Add a -Lm.... commandline argument
Add a -Model... commandline argument
and run your script
this shall process the image

Use OpenEV to digitize training sites for bottom typing:
- digitize small and homogeneous training sites for retreiving bottom type signature
Use 4SM to perform bottom typing

add a -Classify... commandline argument
run 4SM to collect bottyom type signatures
run 4SM to clasify the shallow water areas
this writes a U8 channel to map the bottom typing
then if desired run 4SM to apply a modal filter
this writes a U8 channel to map a cleaner bottom classification

Use 4SM to investigate Sea Truth

Add a -RegressZZ... commandline argument
and run your script: this shall
- write a datbase_RegressZZ.txt text file
- write andd then run a datbase_RegressZZ.sh script
- and finally display a datbase_RegressZZ.eps

Use 4SM to plot sea truth profiles

Add a -ProfileZZ... commandline argument
and run your script

Use 4SM to plot profiles

Add a -ProfileAB... commandline argument
and run your script