Notebooks

Five notebooks have been retained for the processing of transit networks:

  • 01_build_transit_network_gtfs.ipynb
  • 02_build_transit_network_remix.ipynb
  • 03_build_transit_network_rsx.ipynb
  • 04_build_transit_network_from_stm.ipynb
  • 05_export_transit_network_to_emme.ipynb

The user chooses one of the “build_transit_network” notebooks depending on the format of the data, and then the “export_transit_network” notebook.

Files Created

The “build_transit_network” notebook creates/updates the following inputs in ..\schemas\{SCHEMA_NAME}\_emme_inputs\_generated_inputs:

  • networks_transit_itineraries.csv
  • networks.gpkg

The “export_transit_network” notebook creates the following inputs in ..\schemas\{SCHEMA_NAME}\_emme_inputs\_transit_lines:

  • Extra_lines.csv (Headways for Frequency Assingment only)
  • tp_lines_{Time_Period}.txt

The “export_transit_network” notebook updates the following inputs in ..\schemas\{SCHEMA_NAME}\_emme_inputs\_generated_inputs:

  • networks.gpkg - for spatial data
  • schema_database.sqlite - for non-spatial data

Details

01_build_transit_network_gtfs.ipynb

This notebook outlines the process of generating a transit network suitable for modelling in PTM from a GTFS (General Transit Feed Specification) source.

Steps involved:

  1. Setup User Defined Variables: Sets up variables such as schema name, GTFS source folder, modes and stop types to be processed, and whether to clear database tables at the end.
  2. Setup and Initialisation: Runs setup scripts and initialises schema and logging. Defines model parameters and paths.
  3. Import GTFS Data: Reads GTFS files (stops, routes, trips, stop_times, calendar, calendar_dates, agency, shapes) into dataframes. Converts stops data into a GeoDataFrame for spatial operations.
  4. Read Reference Data: Imports reference dates and time periods from CSV files and database tables. Processes time periods to standardise and calculate time period spans.
  5. Process GTFS to Service Schedule: Converts calendar and calendar dates to datetime format. Filters service IDs based on reference dates and removes excluded service IDs. Finally, builds a service schedule by merging GTFS data with time periods and stops.
  6. Create networks_transit_lines: Builds transit lines table by processing GTFS data and appending mode and vehicle codes and converts the result to a GeoDataFrame.
  7. Create networks_transit_itineraries: Matches stops to nodes in the physical network and includes the option for stop to node overrides. Builds transit itineraries table using the matched stops and service schedule.
  8. Save Locally: Exports the transit lines and itineraries to local files, ensuring no duplicates and adhering to schema requirements.

The notebook includes detailed logging at each step to track progress and ensure data integrity.

02_build_transit_network_remix.ipynb

This notebook generates a transit network suitable for modelling in PTM from a REMIX source. Steps involved in this notebook largely follows the same steps as the ‘01_build_transit_network_gtfs.ipynb’.

  1. Rename Line IDs: Renames line IDs to make them more readable or to align with external models immediately prior to being saved locally.

03_build_transit_network_rsx.ipynb

This notebook is designed to build a transit network from a supplied Railsys (RSX) file.

Steps involved:

  1. Setup User Defined Variables:
SCHEMA_NAME: Specifies the schema name for the transit network.
RSX_SOURCE: Folder name containing the GTFS data.
LINE_SEGMENTATION: Level of aggregation for each transit line. (trip_id for timetable models and shape_id for frequency models)
MODE_STOP_TYPES: Modes and corresponding stop types to be processed.
CLEAR_TABLES: Boolean to clear database tables at the end.
TIME_BINNER: Method to allocate to time periods.
CREATE_TRIP_ID: Boolean to create a custom trip ID.
IS_POST_CRR: Boolean indicating if the timetable is for post-CRR implementation.
EXCLUDED_STATIONS: List of stations to be excluded within the RSX timetable.
  1. Setup and Initialisation: The notebook sets up the schema and logs as well as imports necessary time periods, modes, and vehicles from CSV files.
  2. Reading RSX Data: The RSX timetable data is read and parsed using an xml Element Tree. The notebook builds the transit network and service schedule from the RSX data.
  3. Processing Timetable Data: The notebook processes the timetable data to build a service schedule. It handles special cases like the post-CRR implementation adjustments.
  4. Building Transit Network and Service Schedule: The service schedule is built by processing RSX data. It includes steps to handle empty trains and bin services into time periods.
  5. Creating Transit Lines and Itineraries: The notebook creates transit lines and itineraries based on the processed schedule. It matches stops to nodes and applies any necessary overrides.
  6. Save Locally: The final transit lines and itineraries are validated and exported to a GeoPackage and CSV files.

The notebook includes detailed logging at each step to track progress and ensure data integrity.

04_build_transit_network_from_stm.ipynb

This notebook is designed to build a transit network from a Strategic Transport Model (STM) run such as from the SEQSTM.

Steps involved:

  1. Setup User Defined Variables: Defines schema name, STM model type, STM folder, and lists of lines and extra lines files to import. This notebook also sets a flag to clear tables if necessary.
  2. Ingest from Lines / Extra Lines: Ingests lines and extra lines files based on the STM model type. Processes and cleans the data, handling specific columns and markers and finally concatenates all ingested lines and extra lines into dataframes.
  3. Create Transit Line Geometry Object: Reads network nodes from a GeoPackage file and creates simple shapes for transit lines by merging with nodes data.
  4. Format Dataframes to Lines and Itineraries Tables: Calculates additional attributes like length in kilometres and then groups and aggregates data to create transit lines and itineraries. This notebook also handles special cases for events models and timetables and renames and formats columns to match required schema.
  5. Save Locally: Exports the transit lines and itineraries to a GeoPackage and CSV file respectively.

The notebook includes detailed logging at each step to track progress and ensure data integrity.

05_export_transit_network_to_emme.ipynb

The notebook is designed to export a transit network from a PTM database to EMME transaction files suitable for immediate import to PTM. It supports various models, including PTM (Frequency), PTM (Timetable), PTM (Olympics).

Steps involved:

  1. Setup User Defined Variables: Setting up schema names and GTFS data sources.
  2. Update Operator IDs (for GTFS Only): Obtaining operator IDs if GTFS data sources are used. This function is rarely used.
  3. Import Lines and Itineraries: Importing transit lines, itineraries, and time periods from specified files.
  4. Check Itineraries: Ensuring each route has at least two unique nodes.
  5. Perform Modifications: Applying modifications such as:
    • route deletions
    • vehicle amendments
    • headway adjustments
    • routing changes including adding dummy nodes to force routing.
  6. Prepare Transit Attributes: Preparing additional attributes for lines and stops, such as flags for high-frequency and TUAG lines.
  7. Export to Transaction Files: Exporting the prepared data to EMME transaction files and logging the completion.

The notebook ensures data integrity by checking for unique nodes in itineraries, applying necessary modifications, and exporting the final transit network data in a format suitable for EMME.