You should do your bookkeeping in a normal database, but the models themselves usually need something specialized. Ideally keep them in cache/RAM, because if you have to hit the disk you'll probably get beaten to execution by another HFT. If the data set is too large to keep in RAM (and you can't afford to just buy more computers), then page out portions to flash and mmap them back in when you need to. (Ironically, this is sort of how programs were constructed in the 1970s before the days of virtual memory, filesystem abstractions, and file formats.)
Absolutely. Our models didn't look anything like most programs I have worked on elsewhere in terms of architecture etc. But the execution/bookkeeping records just went into a conventional sql db out of band from the main routing/strategy logic.
