Network Security · 2024 · Fortinet

Fortinet OT Greenfield Reference Architecture

  • Fortinet FortiGate 60F
  • Fortinet FortiGate Rugged 70F-3G4G
  • Fortinet FortiGate-VM
  • Fortinet Secure SD-WAN
  • Fortinet FortiManager
  • Fortinet FortiAnalyzer
  • FortiGuard OT Security Service
  • AWS VPC
  • IPsec VPN
  • Cellular WAN (LTE)
  • Starlink
Role
Network Architect
Period
2024
Stack
  • Fortinet FortiGate 60F
  • Fortinet FortiGate Rugged 70F-3G4G
  • Fortinet FortiGate-VM
  • Fortinet Secure SD-WAN
  • Fortinet FortiManager
  • Fortinet FortiAnalyzer
  • FortiGuard OT Security Service
  • AWS VPC
  • IPsec VPN
  • Cellular WAN (LTE)
  • Starlink
Evidence
1 document
Standards & frameworks
  • ISO/IEC 27001 Informed by
  • Purdue Model Informed by
  • NIST 800-82 Informed by

Scenario

A regional refinery and pipeline operator required a greenfield OT network architecture spanning a refinery, distributed pipeline sites, third-party control environments and cloud-hosted infrastructure. The surrounding estate had grown organically across refineries and remote terminals with no consistent architectural standard, mixing network, firewall, switching and cellular equipment from many vendors. This region was therefore built as the reference architecture for a broader standardization programme — covering secure hub-and-spoke connectivity, OT segmentation, resilient multi-path WAN, controlled third-party communication, centralized Fortinet management and an AWS-hosted management and security environment, in a form that could be adapted to further regions and to newly acquired sites.

Documents

Ten-sheet greenfield architecture set for a regional industrial operator: hub-and-spoke SD-WAN, refinery and pipeline OT segmentation, ruggedized remote sites on cellular and satellite WAN, third-party control-centre connectivity, and AWS VPC segmentation with centralized FortiManager and FortiAnalyzer management. Sheet 1 is the anonymization notice.

  • Design
  • 10 pages
  • 3.4 MB
  • Uploaded Oct 31, 2024
  • Vendor Fortinet
  • Category Network Security

Technologies

  • Fortinet FortiGate 60F
  • Fortinet FortiGate Rugged 70F-3G4G
  • Fortinet Secure SD-WAN
  • Fortinet FortiManager
  • Fortinet FortiAnalyzer
  • AWS VPC
  • IPsec VPN
  • Cellular WAN (LTE)
  • Starlink

Notes

Designed, standardized and deployed, with the full set produced for handoff to support engineers. Sheet 1 is an anonymization notice: customer names, addressing, relationships and configurations were deliberately removed or generalized, and some details may therefore read as incomplete or technically inconsistent. Treat the set as a representative example of the architecture and segmentation methods, not as an as-built specification.

Role

  • Greenfield OT network architecture and design
  • OT segmentation and security-zone strategy across refinery and pipeline environments
  • Hub-and-spoke SD-WAN and IPsec connectivity design
  • HLD and LLD development for hub, spoke, third-party and cloud domains
  • Production of the standardized drawing set used for engineering handoff
  • Definition of the reusable technical standards — addressing conventions, security-policy structure and SD-WAN settings — applied across sites rather than per site
  • Lab validation of the PLC-cabinet and remote-site topology ahead of production deployment
  • Establishment of the deployment and migration methodology used to bring sites onto the standard

Challenges — Why

  • Operational systems were distributed across a refinery and eleven geographically separated pipeline sites containing PLCs, flow-computing systems and other OT equipment.
  • Product and crude operational environments required logical separation while retaining controlled communication with shared infrastructure and authorized control systems.
  • Remote sites required resilient connectivity across several available WAN paths, including terrestrial internet, cellular and satellite.
  • Third-party control environments required access to operational systems without being treated as part of the trusted internal OT environment.
  • Cloud-hosted management and security services had to integrate with OT and enterprise connectivity while keeping management, transit, DMZ and untrusted boundaries separate.
  • Public-safety cellular requirements differed by carrier — FirstNet operates on Band 14 and Verizon Frontline on Band 13 — constraining which ruggedized platform could satisfy which site.
  • No consistent architectural standard existed across the wider estate, leaving heterogeneous vendor equipment, recurring connectivity failures, limited centralized visibility and site visits for routine recovery.

Deliverables

  • Cover sheet and drawing index for the standardized set
  • Hub FortiGate call/spec sheet
  • Spoke FortiGate call/spec sheet
  • Hub-and-spoke architecture views, original and expanded
  • Spoke-site expanded low-level design
  • Third-party control-centre high-level design
  • Main refinery high-level design
  • AWS VPC segmentation low-level design
  • Future-state bills of materials and replacement strategy for legacy networking and cellular equipment
  • Configuration, addressing and deployment standards reusable across sites
  • Migration guidance and operational handoff documentation
  • Recommended sequence for modernizing further regions by business criticality and production impact

Outcomes

  • Established a segmented Fortinet OT architecture separating operational zones while preserving controlled communication between authorized systems.
  • Delivered a hub-and-spoke SD-WAN and IPsec architecture connecting refinery infrastructure, distributed OT sites and authorized third-party control environments, with the hub built as a high-availability FortiGate pair.
  • Specified resilient remote-site connectivity across cellular and satellite WAN in addition to terrestrial paths, with ruggedized dual-power, dual-SIM hardware deployed in high-availability pairs at all eleven terminals.
  • Extended the architecture into AWS with separated management, DMZ, transit and untrusted domains behind centralized FortiManager and FortiAnalyzer.
  • Produced a reusable HLD/LLD set covering hub, spoke, third-party and cloud integration, standardized for handoff to support engineers.
  • Validated the design in a lab that reproduced the PLC-cabinet and remote-site topology before any production deployment.
  • Delivered the first region in roughly three to four months against an expected five to six, with the standardized, lab-tested design absorbing most of the difference.
  • Established a repeatable regional model — architecture, security, connectivity, management and governance — that could be applied to further regions and to newly acquired sites without redesign.

Full write-up

The estate before the design

The operator’s OT footprint had grown the way most industrial estates grow: one site at a time, each solved with whatever was available and defensible on the day. Refineries and remote terminals ran a mixture of network, firewall, switching and cellular equipment — Cisco, Peplink, Digi International, Sierra Wireless, Cradlepoint, Netgear, Dell and others — with no consistent architectural standard binding them together.

The operational consequences were the familiar ones. Connectivity failures recurred without a common diagnostic path. Centralized visibility was limited, because there was no single place from which the estate could be seen. Environmental conditions took equipment out at sites that had never been designed for them. Remote-access methods differed by site, which made secure access a per-site negotiation rather than a property of the network. And often enough, restoring a site meant dispatching somebody to it for work that should never have required a vehicle.

Why this region became the reference

A newly acquired refinery and its eleven associated pipeline and terminal sites were selected as the first greenfield build. That decision mattered more than it might appear. The alternative paths were to keep absorbing each environment on its own terms, or to begin a large-scale hardware replacement before anyone had agreed what the target actually was. Both spend money without accumulating a standard.

Treating this region as the reference architecture meant the first deployment had to carry a second obligation: everything produced for it had to be reusable somewhere else. I was brought in to develop that architecture, define the technical standards, validate the solution, and establish the deployment and migration methodology that would carry it into subsequent regions and into sites acquired later.

The architecture

The design is a modular hub-and-spoke built on Fortinet Secure SD-WAN, with encrypted VPN connectivity, high-availability options, security-zone separation and granular segmentation between the refinery, crude, product, third-party and management environments.

At the hub, a high-availability pair of FortiGate 60F appliances — Router A and Router B — rather than a single gateway, with a dedicated HA link and a separate security segment. The representative drawing set carries the platform’s specifications alongside the design: 1.4 Gbps IPS throughput, 6.5 Gbps IPsec VPN throughput, active-active and active-passive HA, and Secure SD-WAN.

At the terminals, ruggedized hardware without exception. All eleven remote sites use the FortiGate Rugged 70F-3G4G (model FGR-70F-3G4G), deployed in high-availability pairs, with dual DC power inputs, dual SIM, GPS, integrated LTE and a hardware bypass path. The ruggedized platform was not a preference; it was the entry requirement for the industrial cabinets and geographically dispersed sites this estate runs on. Where the facility supported it, cabinets were designed around redundant WAN paths, high-availability firewall configurations and independent electrical feeds, so that no single carrier, firewall or power path could take a terminal off the air by itself.

Connectivity, and the carrier-band constraint

Transport was designed to be interchangeable. Depending on what a site could actually get, the architecture combines terrestrial broadband or carrier circuits, Starlink, LTE and public-safety cellular. The hub, for instance, takes fibre on WAN1 and Starlink on WAN2.

Public-safety cellular is where the design had to be explicit rather than aspirational. FirstNet, Built with AT&T operates on Band 14. Verizon Frontline runs on Band 13. The FGR-70F-3G4G in the configuration under evaluation supported the Verizon requirement but did not provide Band 14, so it could not satisfy FirstNet.

That limitation is written into the drawing set rather than worked around. FirstNet compatibility was not required for the initial region, and holding the region while the platform caught up would have cost schedule for no operational gain. The more important protection was structural: because access technology is decoupled from the rest of the architecture, a site can change how it reaches the network without anything else being redesigned. Fortinet has since expanded its rugged cellular portfolio, and that change lands as a hardware swap rather than an architectural revision — which is exactly the outcome the decoupling was for.

Third-party access

External organizations responsible for systems such as historian and control-support platforms were not given generalized access to the OT environment. Connectivity reaches only the required networks and services, with security policy and segmentation defining precisely which systems may communicate. The set documents dedicated third-party control-centre connectivity alongside separate refinery, crude, product, security and DMZ segments, rather than establishing unrestricted site-to-site reachability and then trying to constrain it afterwards.

Centralized management in AWS

The supporting cloud environment was designed with the same segmentation discipline as the plant network. FortiManager provides centralized configuration and policy management; FortiAnalyzer provides centralized logging, analytics, reporting and security visibility. The AWS architecture separates management, DMZ, transit, production and untrusted segments, fronted by a FortiGate-VM instance, with administrative access arriving through a controlled jump-host path rather than management workloads being exposed as ordinary cloud resources.

Security posture and frameworks

Security was part of the architecture rather than an overlay applied once connectivity worked. The design was informed by the Purdue Enterprise Reference Architecture, NIST SP 800-82 guidance for industrial control systems and OT security, and ISO/IEC 27001 information-security management principles.

That phrasing is deliberate. The architecture was developed using principles informed by those frameworks. It is not a claim of formal compliance or certification, because a representative architecture package cannot substantiate one.

FortiGate subscriptions included the FortiGuard OT Security Service, providing OT-specific detection, industrial protocol and application visibility, IPS protections, vulnerability correlation and virtual patching. The project documentation specified FortiCare 24x7 Comprehensive Support under the customer’s enterprise support agreement; Fortinet’s current equivalent nomenclature is FortiCare Premium Support.

Building for repeatability

The requirement that separated this from a competent site build was that it had to be reproducible by someone else.

Configurations were built and validated in a lab that reproduced the PLC cabinet and remote-site topology before anything reached production. Addressing conventions, security-policy structures, SD-WAN settings, deployment procedures, bills of materials and handoff documentation were all developed as standards rather than as artifacts of one site. Subsequent terminals and regions deploy from those templates, with site-specific elements added or removed according to connectivity, environment, operational role and business requirements.

Delivery

The first region had been expected to take roughly five to six months to deploy and validate. With the architecture standardized, lab-tested and supported by the wider implementation team, deployment and verification completed in approximately three to four months — hardware availability accounting for a meaningful share of what remained.

The more durable result was what that made possible. Work originally framed as a potentially multi-year modernization became a repeatable regional architecture that could be deployed progressively, sequenced against budget, operational criticality and business priority rather than attempted all at once.

What was handed over

Beyond the first implementation: architecture drawings, HLD and LLD documentation, configuration and deployment standards, migration guidance, future-state bills of materials, replacement strategies for the heterogeneous legacy networking and cellular equipment, operational handoff documentation, and a recommended sequence for modernizing further regions by business criticality and production impact.

What the engagement produced was not a refresh of aging network equipment. It was a repeatable IT/OT architecture, cybersecurity, connectivity, management and governance model — one the organization could apply to further regions, and to refinery and terminal sites acquired after the fact, without starting the architectural argument over each time.

About this artifact

The accompanying package, OT Deployment — Regional Fortinet Design — Greenfield, is a representative version of this architecture. It documents hub-and-spoke connectivity, refinery and pipeline segmentation, controlled inter-site and third-party communication, Secure SD-WAN, ruggedized FortiGate deployment, OT security services and the AWS-based FortiManager/FortiAnalyzer management architecture.

It has been intentionally anonymized and modified. Names, addressing, relationships and certain design elements were generalized or altered to prevent identification of the customer or reconstruction of the production environment, and it should be read as a representative architecture artifact rather than an unmodified as-built.