collapse

* Forum Staff

PeterC admin PeterC
Administrator
admin webmasterOPS
Administrator

Author Topic: The input CSV files  (Read 236 times)

PeterC

  • Administrator
  • Member
  • *****
  • Posts: 382
  • Passionate about my one-place study
    • View Profile
    • Holywell-cum-Needingworth History
The input CSV files
« on: 15 January 2017, 11:59:19 »
Note that this is mostly for technical information.

For each OPS in M4OPS

Input files
All files are in csv format using semi-colon ";", rather than comma, as the separator. No fields can have semi-colons within them, but they can have commas. (On Peter's PC all csv files are within eg \OPS\HcN Holywell-cum-Needingworth\FLG\.)

Every file has field names in its first row, and these field names must be as specified (in the correct case). No quotes are needed for field names or values. Any null or blank values should be left null, ie represented as ;;.

Any blank lines, any starting with the word “Comment”, and any starting with the first field name (thereby allowing a repeat of the first row of field names), are ignored. If you use Excel for accuracy then ensure that the delimiter is set to ";".

These csv files are used to generate geojson and json files, which are what M4OPS to use directly as its input.

The required files are:
  • OPS.csv - a single row with the parameters of the OPS
    • It must have the following fields
      • OPSCode - short code (this is often 3 upper case letters)
      • OPSName - the name of the OPS (usually mixed case)
      • Area – of the OPS, eg ENG or SCO (if otherwise then Peter will let you know what to use)
      • Lon – longitude of the centre of the OPS (in decimal degrees - known as EPSG:4326)
      • Lat – latitude of the centre of the OPS (ditto)
      • password - requires to be entered before any updating
    • It can have the following fields
      • OPSDescription – appears when you click on the heading, can include abbreviations (default is “No description available”)
      • ContactEmail – ditto (no default), and where the Contact button email goes to
      • Website – ditto (no default)
      • Zoom – starting zoom (default is 16)
      • Rotation – starting rotation in radians clockwise, default is 0 (ie North at the top)
      • nFeatureLayers – number of feature layers to allow/show (default 0, maximum 3)
      • minYearBound - minimum year to allow (default 700AD)
      • maxYearBound - maximum year to allow (default the current year)
      • minYear -  the starting lower value for the range of the lower time slider (default minYearBound)
      • maxYear -  the starting upper value for the range of the lower time slider (default maxYearBound)
      • IconsFolder – the icons folder to use if specific to this OPS (M4OPS default is icons/default-icons/)
      • minx (Not implemented yet - becomes extent, with miny, maxx, maxy - must all be there or all absent)
The optional files are:
  • LayerDefs_Feature.csv - a list of all feature (vector) layers specific to this OPS
    • It must have the following fields
      • layertype - always Vector
      • category - used to group the layers, the basis of the category drop-downs
      • title - the name of the layer (and is used when specifying a parameter for preloading a layer)
      • url - specification of where to get the layer from:
        • a file name for a GeoJSON file
      • attribution – to be shown when the layer is displayed, can be an abbreviation
    • It can have the following fields
      • csvname - the name of the csv file (without the .csv) if it is to be included in any compiling
      • candownload - either TRUE if the layer can be downloaded eg for printing, or null if not (default is null)
      • mapkey – used when the key button is clicked, can be either:
        • the file name for a JPG file in the OPS's images folder, or
        • the url of a web page
      • minx (Not implemented yet - becomes extent, with miny, maxx, maxy - must all be there or all absent)
      • shorttext - defines the name of the field in this feature layer that is the equivalent to "shorttext", eg when hovering over features (no default)
      • description - defines the name of the field in this feature layer that is the equivalent to "description", eg when displaying in the panel (no default)
      • fl_Ncols – the number of columns in the feature list (default is 1)
      • flhead_col1 – the heading for the first column (default is blank)
      • flhead_col2 – the heading for the second column (default is blank)
      • flhead_col3 – the heading for the third column (default is blank)
      • layerdescription - html text shown when you click on the Description button (?), can include abbreviations
  • LayerDefs_Other.csv - a list of all other layers specific to this OPS, including raster Tiles
    • (Note that the general DefinedLayerDefs_Other.csv file has exactly the same specification, except that Defined layertype is not allowed)
    • It must have the following fields
      • layertype - one of
        • Tile - either from a map server, or as prepared by maptiler and served by TileServer
        • VectorTile - from a map server (not implemented yet)
        • WMTS - from a Web Map Tile Service/url]
        • Group (made up of any number of other layers)
        • Defined (picked up from DefinedLayerDefs_Other.csv)
      • category - used to group the layers, the basis of the category drop-downs
      • title - the name of the layer
        • (this is used when referring to a layer eg for specifying a parameter for preloading, or picking up a (pre-)Defined layer)
      • Note that Defined layertypes need only these three fields, and not even nulls and separators for the other fields
    • It must also have the following fields (unless it is a Defined layertype)
      • url - specification of where to get the layer from, either:
        • a full url (if served by a map server), or
        • a folder name (if served by TileServer), or
        • (for WMTS layertypes) a full url of a Capabilities file, or
        • (for Group layertypes) a comma separated list of layer titles that are to be included in the group, eg Lidar Coldhams Field,Lidar Holywell Centre 1,Lidar Holywell Centre 2
        • note that if preferred this can be provided as an array of quoted layer titles, eg ["Lidar Coldhams Field","Lidar Holywell Centre 1","Lidar Holywell Centre 2"]
      • attribution – to be shown when the layer is displayed, can include abbreviations (default null)
      • folder - either:
        • (for Tile layertypes) where the url folder is within the TileServer folder (default is null), or
        • (for WMTS layertypes) the WMTS layer name
    • It can have the following fields
      • donotshow - TRUE if the layer should NOT be listed as itself in the drop-down (usually because the layer is part of a layer Group) (default is false -  it will be listed)
      • candownload - TRUE if the layer can be downloaded eg for printing (default is false)
      • useproduction - TRUE if the ShowMapsDev version should use the folder within the ShowMaps setup (default is true)
      • mapkey – used when the key button is clicked, can be either:
        • the file name for a JPG file in the OPS's images folder, or
        • the url of a web page
      • projection – not used yet (EPSG:3857 or EPSG:4326 at present)
      • minZoom – the minimum zoom level that tiles are available for (default is 0)
      • maxZoom – the maximum zoom level that tiles are available for (default is 18)
      • tilePixelRatio – usually 1 (not implemented yet)
      • origin - (not implemented yet) the tiling origin (topleft = OpenGIS, M4OPS default for tileserver, and MapTiler default)
        • (bottomleft was tried, but tileserver can only do top left)
      • minx (becomes extent, with miny, maxx, maxy – must all be there or all absent)
      • layerdescription - html text shown when you click on the Description button (?), can include abbreviations
  • Features.csv - a list of all the features known to the OPS (which can then be referred to by Feature Layers)
    • It must have the following fields
      • featureid - string value by which this feature is referred
      • shorttext - name of the feature, displays on the map when resolution is OK, and in the display panel
    • It must also have one of the following geometries
      • a pair of coordinates corresponding to a point
        • Lat(Y) - numeric (eg 52.32)
        • Lon(X) - numeric (eg -0.03)
      • a geometry specification, comprising:
        • GeomType - Point, LineString, or Polygon
        • GeomCoords - a CoordinateString, eg:
          • for a Point: [-0.03, 52.32]
          • for a LineString: [[30, 10], [10, 30], [40, 40]]
          • for a Polygon: [[30, 10], [10, 30], [40, 40], [30, 10]]
        • always Lon/Lat in decimal degrees - known as EPSG:4326
        • see https://en.wikipedia.org/wiki/GeoJSON for other examples
      • Note that the following are the same point:
        • Lat(Y)=3 and Lon(X)=5
        • GeomType=Point and GeomCoords=[5,3][/i]
      • Note also that we use the field name Lat(Y) for latitude to remind ourselves that in mathematics this is the Y-axis, and similarly Lon(X) as longitude is the X-axis
    • It can have any of the following fields (these correspond to those listed under the PROPERTIES for the GeoJSON file, and become the defaults for all Feature Layers)
      • description - html that displays in the display panel, (default is shorttext)
      • datestart - date from which this feature is to be shown in ISO Format - YYYY-MM-DD, eg 1888-03-25, or 1910 (default 0000-01-01)
      • dateend - date after which this feature is not to be shown (ditto) (default 9999-12-31)
      • internalnote - for internal use eg as a reminder – not displayed anywhere
      • fl_col1 - used as the value in the first column in the feature list, if undefined shorttext is used
      • fl_col2 - used as the value in the second column in the feature list
      • fl_col3 - used as the value in the third column in the feature list
      • textforsort - used for sorting the feature list, if undefined then the value of fl_col1 is used
      • image - the relative filename of an image associated with the Feature (in the Images subfolder)
      • pin - pin number - see the available pin styles
      • symbol - name of the icon to use - see the available A-M and N-Z icons
      • color - colour of the circle - see the full range of colours available (default is red)
      • radius - size in pixels of the (filled) circle (default is 5)
  • People.csv - a list of all the people known to the OPS (which can then be referred to by Feature Layers)
    • It must have the following fields
      • personid - string value by which this person is referred (becomes xrefid in the JSON file)
      • shorttext - name of the person (becomes xl_col1 in the JSON file)
    • It can have any of the following fields
      • textforsort - if undefined xl_col1 (shorttext ) is used
      • xlhint -  the hint on hover over the person in the list (eg Birth)
      • datestart - date (eg Birth) from which this person is to be shown in ISO Format - YYYY-MM-DD, eg 1888-03-25, or 1910 (default 0000-01-01)
      • dateend - date (eg Death) after which this person is not to be shown (ditto) (default 9999-12-31)
      • image - the relative filename of an image associated with the Person (in the Images subfolder)
  • Abbreviations.csv - a list of all the abbreviations specific to this OPS
    • It must have the following fields
      • abbcode - the text to be replaced (by convention usually starts and ends with '#')
      • abbhtml - the text by which it is to be replaced
  • Exclusions.csv - a list of all the layers from the Area of the OPS that should be excluded
    • It must have the following fields
      • category - of the layer to be excluded
      • title - name of the layer to be excluded

For each Feature Layer

The required files  (using the example of “Censuses”) are:
  • Censuses.csv - a list of all the features known to the OPS used in this Feature Layer
    • It must have either:
      • featureid - Required, string, corresponds to the featureid in Features.csv, or
      • one of the geometries listed above under Features.csv
    • It can have any of the other fields listed above in Features.csv (except the geometry - see above), and, if present, these values then override the values in Features.csv
    • Note that if no featureid is given then it must have a shorttext
    • Note that if a featureid and an override shorttext is given then you may also need to give an override fl_col1
The optional files are:
  • Censuses_Events.csv - a list of all the events listed for all the features used in this Feature Layer
    • It must be sorted by featureid, and then in the order you wish them to appear (eg by evdateend)
    • It must have the following fields
      • featureid - Required, string, corresponds to the featureid in Features.csv
      • evshorttext - the short summary heading for the event
      • evdatestart - date from which this event is to be shown in ISO Format - YYYY-MM-DD, eg 1888-03-25, or 1910 (default 0000-01-01)
      • evdateend - date after which this event is not to be shown (ditto) (default 9999-12-31)
      • evdescription - a full html description of the event
    • Note that this requires a featureid and must therefore be in Features.csv
  • Censuses_People_Events.csv - a list of all the events listed for all the people and features used in this Feature Layer
    • It must be sorted by personid, and then in the order you wish them to appear (eg by evdateend)
    • It must have the following fields
      • personid - Required, string, corresponds to the personid in People.csv
      • featureid - Required, string, corresponds to the featureid in Features.csv
      • evshorttext - the short summary heading for the event
      • evdatestart - date from which this event is to be shown in ISO Format - YYYY-MM-DD, eg 1888-03-25, or 1910 (default 0000-01-01)
      • evdateend - date after which this event is not to be shown (ditto) (default 9999-12-31)
      • evdescription - a full html description of the event
  • Censuses_People_Specs.csv - contains the fields specifying the person and results lists
    • It can have the following fields
      • XRefName - (default People)
      • fnresultsHead - the html heading text for each Person (where the text "xrefid" is replaced by the personid)
      • fnresultsBody - the html body of text for each Person (ditto)
      • fnresultsevHead - the html heading text for each event (ditto)
      • fnresultsevBody - the html body of text for each event (ditto)
      • xl_Ncols - the number of columns to be shown in the People list (default 1)
      • xlhead_col1 - the heading for the first column of the People list (default is blank)
      • xlhead_col2 - ditto column 2
      • xlhead_col3 - ditto column 3
   
« Last Edit: 14 March 2017, 10:20:02 by PeterC »

 

mapping4ops.org is a Society for One-Place Studies project supported by Grassroots Giving from Skipton Building Society
Glossary | BBCodes | Feedback