This is highly insightful for OPs original question:

"Is it possible to just scale up the hardware"

In 1999 I was doing NAT and simple stateful firewall and some stereotypical appliance functions (DHCP, DNS, NTP) on a 25 MHz 486 desktop repurposed into my firewall and when maxing out the 1 meg dsl line I was running around 20% CPU, so I estimated the hardware wouldn't be limiting until 5 megs or so. I've upgraded a couple times since then.

Then again if you enable enough logging and packet inspection you can probably kill a brand new top of the line server on a 56K SLIP connection with one user.

There are certain pitfalls... at a glance comparing my 486 first firewall to a modern rasp pi, the pi should win, but the pi connects ethernet over its usb with pretty icky limits and latency. So my old 25 MHz 486 would probably crush a rasp pi acting in that role despite the orders of magnitude disparity in CPU speed.

In summary based on lots of experience, the variation in what you're trying to do, influences the "power" required, by several orders of magnitude more than the speed of the connection or number of users.

There is an interesting analogy in supercomputing that no matter how big the machine its not much of a challenge to submit an algo that scales poorly such that a modest appearing increase can crush it, see traveling salesman problem etc. In a similar way you could probably run a firewall on an embedded 486 appliance, although it would be trivial to configure a fw to absolutely crush a top of the line modern server no matter how much money was spent on it.

Is this a limitation of VyOS or the closed-source offload/acceleration driver?

For Intel hardware, DPDK improves performance even in virtualized environments, http://rishidot.com/blog/cloudcomputing/intel-dpdk-and-cloud... & http://events.linuxfoundation.org/sites/events/files/slides/...