.. _project structure:
LUPTAI Project Structure
========================
A LUPTAI model consists of a number of files and directories, however all the essential input and output
data lives within a single spatialite file for each modelled scenario that is being analysed.
The LUPTAI QGIS structure inherits a degree of similarity from the prior ArcGIS plugin.
The typical project structure looks similar to the following:
| Projects
| ├── 2021_SEQ_model
| │ ├── Model.qgz
| │ ├── Scenarios
| │ │ ├── base_model.sqlite
| │ │ └── raw_gtfs_pt_model.sqlite
| │ └── ModelData
| │ ├── base_model
| │ └── raw_gtfs_pt_model
| │
| ├── luptai_sample_project
| │ ├── Model.qgz
| │ ├── Scenarios
| │ │ ├── base_scenario.sqlite
| │ │ └── Story_bridge_removal.sqlite
| │ └── ModelData
| │ ├── base_scenario
| │ └── Story_bridge_removal
|
The ``Scenarios`` folder and the spatialite (sqlite) files within store all the data comprising a scenario, along with
the results of any model runs completed with that scenario. The ModelData folder contains the raw unparsed model
input and output in an xml interchange format. It is not interacted with in general use.
Any project or scenario created through LUPTAI QGIS will automatically propagate the correct structure. Note that as a
scenario primarily consists of a single file, this can be copied between projects freely (unlike the ArcGIS version
which contained a project tracker). You can also rename a scenario, however the results tables contained within the
spatialite file will not be updated correspondingly (which may or may not be important depending on your use case).
.. _scenario structure:
LUPTAI Scenario Structure
=========================
Now that we have seen what a luptai project looks like on disk, we examine the luptai scenario format. This may be
useful if you are looking to import your own custom data into LUPTAI. Throughout this discussion it will be beneficial
to be able to inspect the sample scenarios to examine the data format in more detail. This can be done with an external
tool, such as `DB Browser (sqlite) `_, or directly with QGIS by adding a spatialite layer
and including all the tables within a scenario file (including those without geometry).
.. image:: images/data_structure/load_spatialite_layer.png
:alt: Loading spatialite layers in QGIS.
Here we see a list of all the geometry columns and the tables with no geometry columns. We now cover the format
of the tables intended to be constructed by the user, to aid in the process of importing custom data.
This includes the tables ``network_links``, ``network_nodes``, ``activities_points`` and ``activities_attributes``.
Note that whilst any Coordinate Reference System (CRS) is supported for the import layers, they will be
converted to EPSG:4283 in the scenario spatialite. Layers with invalid CRS will have the CRS EPSG:4326 manually provided
to them, which may or may not produce sufficient results.
Network Links Table
-------------------
The following descibes the network_links table structure within the spatialite file. The columns marked essential
contain information used by LUPTAI and must be provided. The remaining columns are nullable, containing
non-essential metadata inherited from the QBase and LUPTAI ArcGIS data formats.
.. .. csv-table:: ``network_links`` Fields
.. :file: tables/network_links.csv
.. :widths: 20, 10, 50, 20, 10
.. :header-rows: 1
+------------+-------------+-----------------------------------+----------------+---------------+
| | | | | | Used in |
| Column | Data Type | Description | Example Values | | Comp. |
+============+=============+===================================+================+===============+
| fromnodeid | INTEGER | node in ``network_nodes`` | 17708 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| tonodeid | INTEGER | node in ``network_nodes`` | 17709 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| linkid | INTEGER | unique per link | 219119 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| meters | FLOAT | link shape length in metres | 1240.042 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| enabled | INTEGER | boolean flag to ignore from model | 1 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| luptaitype | INTEGER | see below | 5 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| roadtype | INTEGER | see below | 5 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| onewayind | INTEGER | see below | 0 | x |
+------------+-------------+-----------------------------------+----------------+---------------+
| shape | BLOB | linestring geometry | |
+------------+-------------+-----------------------------------+----------------+---------------+
| objectid | INTEGER | auto incremented id | 18400 | |
+------------+-------------+-----------------------------------+----------------+---------------+
| street | VARCHAR(50) | street name | HUCKER ROAD |
+------------+-------------+-----------------------------------+----------------+---------------+
| fromleft | FLOAT | | 0 | |
+------------+-------------+ +----------------+---------------+
| toleft | FLOAT | | 0 | |
+------------+-------------+ +----------------+---------------+
| fromright | FLOAT | | 204 | |
+------------+-------------+ +----------------+---------------+
| toright | FLOAT | | 204 | |
+------------+-------------+ +----------------+---------------+
| alias_name | VARCHAR(50) | Historical [1]_ | HUCKERS ROAD |
+------------+-------------+ +----------------+---------------+
| pid | FLOAT | | 3E+08 | |
+------------+-------------+ +----------------+---------------+
| qual_pnt | VARCHAR(20) | | CDR4M1220100910 |
+------------+-------------+ +----------------+---------------+
| road_id | FLOAT | | 335581 | |
+------------+-------------+ +----------------+---------------+
| locality | VARCHAR(50) | | SPRING CREEK |
+------------+-------------+ +----------------+---------------+
| state | INTEGER | | 3 | |
+------------+-------------+ +----------------+---------------+
| source | VARCHAR(2) | | 2 | |
+------------+-------------+-----------------------------------+----------------+---------------+
RoadType Field
~~~~~~~~~~~~~~~~~
The road type field provides a classification of the type of link. The following comments and encoding table is extracted
from the original LUPTAI technical documentation. `(See Section 5.6.8.2 for more details) <../LUPTAITechnicalDocument.pdf#page=1>`_
Non-walkable links should be removed from the network (manually, this is not done as part of the import).
This will include motorways, on and off ramps etc. This can be done by using the ROADTYPE field from the SDRN.
Some guidelines are in the following table.
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | |
|Value| Classification | Exclude from LUPTAI Walk Network |
+=====+=======================+=============================================================================+
| | | | Yes exclude – no pedestrian access. Note there are a few class |
|1 | Freeways/Motorways | |
| | | | 1 roads which need to be kept in to maintain connectivity of |
| | | | the walk network |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Some network segments may need to be removed. Requires some |
|2 | Highways | | local knowledge to determine if they should be removed. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | |
|3 | Secondary Roads | Do not exclude. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | |
|4 | Local Connector Roads | Do not exclude. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | |
|5 | | Street/Local – only | Do not exclude. |
| | | provides property | |
| | | access | |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Do not exclude. |
|6 | | Notified Private or | | These roads while private or restricted to vehicles, may |
| | | Restricted Roads | | provide pedestrian access especially where they occur within |
| | | | “gated” communities. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Do not exclude – these tracks are likely to be walked by |
|7 | 4WD and Tracks | | pedestrians; however, it is believed that they will occur in |
| | | | locations that would not impact duly on the results of a |
| | | | LUPTAI analysis. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | |
|8 | Malls/Passenger Ferry | Do not exclude except remove Passenger Ferry. |
| | | |
| | | |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Yes exclude – these lines are used to construct some road segment, |
|9 | Construction lines | | e.g. roundabouts which are included under other classification codes. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Yes exclude – these roads either do not exist or do not have |
|10 | | Unconstructed | | pedestrian access. |
| | | and/or Dedicated | |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Yes exclude – to appropriately include vehicle ferries would |
|11 | Vehicle Ferry | | require a time schedules for services and additional capabilities |
| | | | within the LUPTAI model to account for these services. Even so, it |
| | | | is considered that most of these services are located in areas that |
| | | | would not influence pedestrian access within the LUPTAI analysis. |
+-----+-----------------------+-----------------------------------------------------------------------------+
| | | | Do not exclude but using the LUPTAITYPE field identify which are |
|12 | Bikeways/Walkways | | cycle and which are walk paths While bikeways may not be shared with |
| | | | pedestrians, it is assumed in the majority of cases they will be shared. |
| | | | As there is no attribute that identifies shared and unshared bikeways |
| | | | they are not excluded. |
+-----+-----------------------+-----------------------------------------------------------------------------+
|13 | Busways | | Exclude except for a few short links which are classed as busway but |
| | | | have footpaths |
+-----+-----------------------+-----------------------------------------------------------------------------+
LUPTAI Type Field
~~~~~~~~~~~~~~~~~
The luptai type field encodes additional information for walk and cycle links.
The following comments and encoding table is extracted
from the original LUPTAI technical documentation.
`(See Section 5.6.8.5 for more details) <../LUPTAITechnicalDocument.pdf#page=2>`_
For roads the LUPTAITYPE is the same as the SDRN RoadType however for cycle,
walk or shared walk/cycle links these are specified by the following codes.
+------------+---------------------------------+--------------------------------------------------------+
| Value | Description | Comments |
+============+=================================+========================================================+
| | | | Footpaths, cyclists can only travel slowly on these |
| 14 | Footpaths | | links |
+------------+---------------------------------+--------------------------------------------------------+
| | | | These represent larger footpaths or shared paths in |
| | | | parks etc. where cyclists can travel at a |
| | | | reasonable speed but must still be wary of |
| 15 | Shared walk/ cycle paths | | pedestrians |
+------------+---------------------------------+--------------------------------------------------------+
| 16 | Bikeways | Cyclists can travel at maximum speed on these links |
+------------+---------------------------------+--------------------------------------------------------+
| | | | There are some road links which can be cycled on but |
| | | | have no walk access. These should be flagged with |
| 17 | Cyclable but non walkable links | | this code |
+------------+---------------------------------+--------------------------------------------------------+
OneWayInd Field - Network Connectivity
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The OneWayInd Field encodes legal traversal directions for the network links. These are only used for the cycle mode
(as car is not implemented in LUPTAI QGIS).
The following comments and encoding table is extracted
from the original LUPTAI technical documentation.
`(See Section 5.6.8.1 for more details) <../LUPTAITechnicalDocument.pdf#page=1>`_
The network is non-directional for walk mode but is directional for cycling. It uses the OneWayInd field from
the SDRN which has the following values. It is assumed that cyclists follow the same restrictions as cars on
the road network.
+--------------------+-------------------------------+
| | |
| Value | Classification |
+====================+===============================+
| | |
| 0 | Both ways |
+--------------------+-------------------------------+
| | |
| 1 | One way with the vector |
+--------------------+-------------------------------+
| | |
| 2 | One way against the vector |
+--------------------+-------------------------------+
| | |
| 3 | No way |
+--------------------+-------------------------------+
Network Nodes Table
-------------------
The network nodes table contains the nodes of active transport network. The required values identify the position
of the node.
+-----------+-----------+----------------------+----------------+---------------------+
| Column | Data Type | Description | Example Values | Used in Computation |
+===========+===========+======================+================+=====================+
| nodeid | INTEGER | Unique nodeid | 18482 | x |
+-----------+-----------+----------------------+----------------+---------------------+
| easting | FLOAT | | 382101.2809 | x |
+-----------+-----------+----------------------+----------------+---------------------+
| northing | FLOAT | | 6922582.148 | x |
+-----------+-----------+----------------------+----------------+---------------------+
| objectid | INTEGER | Auto incremented id | 18411 | |
+-----------+-----------+----------------------+----------------+---------------------+
| longitude | FLOAT | | 151.8029726 | |
+-----------+-----------+ +----------------+---------------------+
| latitude | FLOAT | Historical [1]_ | -27.81620836 | |
+-----------+-----------+ +----------------+---------------------+
| utm_zone | INTEGER | | 56 | |
+-----------+-----------+----------------------+----------------+---------------------+
| shape | BLOB | Point geometry | | |
+-----------+-----------+----------------------+----------------+---------------------+
.. _activities points table:
Activities Points Table
-----------------------
The activity points table encodes information regarding the activity points. The node id field should
not be populated, it should be set to 0. It will be updated as part of the import process to
reflect the base activity transit network in the scenario. Note that the sentinel values of -1
are used to denote unsnapped nodes. Node id of -2 denotes open space activity type points which were
snapped correctly, but have been excluded from the model as they are in high proximity to other open
space points. The activity types currently supported in LUPTAI QGIS are
['CHILDCARE', 'COMMUNITY', 'HOSPITAL', 'GP', 'PRIMARY SCHOOL', 'SECONDARY SCHOOL', 'TERTIARY', 'OPEN SPACE', 'SUPERMARKET', 'EMPLOYMENT'],
noting that 'EMPLOYMENT' is generally supplied as an attributed activity.
..
There are additional activity types which have previously been supported detailed in the
technical documentation, however the calibration of these types has not been updated.
+----------+--------------+----------------------------+-------------------------+---------------+
| | | | | | Used in |
| | Column | | Data Type | | Description | | Example Values | | Comp. |
+==========+==============+============================+=========================+===============+
| nodeid | INTEGER | node in ``network_nodes`` | 226781 | x |
+----------+--------------+----------------------------+-------------------------+---------------+
| | | | boolean flag to ignore | | |
| enabled | INTEGER | | from model | 1 | x |
+----------+--------------+----------------------------+-------------------------+---------------+
| act_type | VARCHAR(50) | see below | PRIMARY SCHOOL | x |
+----------+--------------+----------------------------+-------------------------+---------------+
| id | INTEGER | unique nodeid | 490 | |
+----------+--------------+----------------------------+-------------------------+---------------+
| feature | VARCHAR(50) | auto incremented id | Primary School | |
+----------+--------------+----------------------------+-------------------------+---------------+
| name | VARCHAR(200) | | Nerang State School | |
+----------+--------------+ +-------------------------+---------------+
| | | | | Qld_Digital Cadastral | |
| label | VARCHAR(400) | Historical [1]_ | | Database | |
+----------+--------------+ +-------------------------+---------------+
| source | VARCHAR(50) | | QSC2020 - Landmark Data | |
+----------+--------------+----------------------------+-------------------------+---------------+
| shape | BLOB | point geometry | | |
+----------+--------------+----------------------------+-------------------------+---------------+
.. _activities attributes table:
Activities Attributes Table
---------------------------
In LUPTAI QGIS, attributed activities are only used for employment and the data format is specified as follows.
Note that the given employment columns are only sample values, any non conflicting column name can be specified.
These are manually associated with the respective employment types through the model run dialogue.
+------------------------+-------------+---------------------------+----------------+----------+
| | | | | | Used in|
| Column | Data Type | Description | Example Values | | Comp. |
+========================+=============+===========================+================+==========+
| emp_blue2016 [2]_ | FLOAT | Blue Collar emp | 14.02 | x |
+------------------------+-------------+---------------------------+----------------+----------+
| emp_white2016 [2]_ | FLOAT | White Collar emp | 14.13 | x |
+------------------------+-------------+---------------------------+----------------+----------+
| emp_total2016 [2]_ | FLOAT | Total emp | 28.12 | x |
+------------------------+-------------+---------------------------+----------------+----------+
| (arb. float cols.) [2]_| FLOAT | see below | | x |
+------------------------+-------------+---------------------------+----------------+----------+
| enabled | INT | | 1 | x |
+------------------------+-------------+---------------------------+----------------+----------+
| act_type | VARCHAR(50) | see act type above | EMPLOYMENT | x |
+------------------------+-------------+---------------------------+----------------+----------+
| nodeid | INT | ``network_nodes`` id | 17008 | x |
+------------------------+-------------+---------------------------+----------------+----------+
| zonecode | BIGINT | zone system id (i.e. SA1) | 30908126121 | |
+------------------------+-------------+---------------------------+----------------+----------+
.. [2] The table supports attributed activities with arbitrary column names however only the three employment type
attributes are exposed by the QGIS plugin. The python notebook api can use the other column names with appropriate
setup.
.. [1] Historic fields for compatibility with LUPTAI ArcGIS