The two books don't map to each other 1:1, but I'd consider a lot of the contents of TAOSSA to be very good foundational content for assessing malware. Much of malware is a practical implementation of faults explored within that book.
I second the reply above. I don't see much overlap between TAOSSA and Security Engineering. And I would absolutely recommend TAOSSA above Security Engineering. I'd almost go as far as to say that if you only read 1 book it should be TAOSSA, but really it depends what you want to do.
TAOSSA also has nothing to do with malware whatsoever. Are you perhaps thinking of a different book?
Also, I would point out that Shellcoders Handbook is extremely dated now. It was excellent in its time, but you won't be able to do much with that anymore. It could still be interesting background, but exploits and exploit mitigations have changed a lot in 13 years. We are talking Windows XP/2003 era.
For all of the flak that Applied Crypto gets, it's actually a very good book for what it is. It's just full of bad/outdated advice. For someone who wants a history lesson and plans to update their knowledge before engaging in malpractice based on what that book teaches, it's still a good read. Cryptography Engineering is a very different style of book. While also good, it doesn't get the reader excited about crypto in the way that Applied Cryptography did. AC also has quite a bit of background and explanation of very basic fundamentals, written for humans as opposed to programmers or math majors, that makes a lot of the ideas behind cryptography more accessible.
Applied Cryptography is pop science. It's a book written breezily and unseriously, a sort of whirlwind tour of a subject its author is fascinated and engaged with, but without rigor. Approached on those terms, it's a fun read. I sure enjoyed it when it first came out.
Unfortunately, it's also a showcase for what can go wrong if you provide too much technical content in a pop science book. What happened with A.C. is similar to what you'd think might happen in a book about home anesthesiology, complete with resources to show you how you'd put someone under from first principles and household chemicals.
On the whole, because it's generally taken by its readers as authoritative and prescriptive, A.C. has done a lot more harm than good.
Cryptography Engineering is imperfect, too. But it was written seriously, accepting and engaging with the fact that readers will use it as a guidebook to serious implementation. It's a much safer book than Applied Cryptography.
Moreover, if you're serious about engaging with crypto in your career, it's a better book. You won't learn much about what goes wrong with crypto from Applied Cryptography, which happily documents a number of weak or even broken constructions and describes protocols and techniques that were obsolete years before the book was published. Cryptography Engineering is aware of much of the cryptographic literature, and almost every chapter concerns itself first and foremost with what goes wrong with crypto constructions. You can read it "inside out" as a sort of first course in attacking cryptosystems.
I'll clarify - I consider Security Engineering and TAOSSA to be landscape volumes - they cover a great deal of depth. Security Engineering is more on the defensive/development side while TAOSSA is on the offensive/assessment side. I agree with you that TAOSSA should be the one to read if you have to only read one.
With regard to malware analysis - TAOSSA teaches source code review to a depth that very few other texts approach in a useful way. Source code review - and the category of vulnerabilities that lend themselves to that assessment - is useful for malware analysis in turn because it teaches the reader what sort of issues malware might try to exploit in a system.
This focuses on the high end of malware - for rote malware analysis and incident response I agree it's not going to be helpful. Like I said, it's more of a foundational work.