It’s a great idea and it works on a very simple principal that should always be held true – reduce attack surface where possible.
The renderer is the ‘exposed code’ in this situation and Chrome runs its renderer at Untrusted Integrity. That means it has no file access. So even if a PDF exploits PDF.js in Chrome it’s trapped in the most restricted part of the browser and there’s really nothing it can do – it can’t read or write anywhere so even in-browser attacks are limited.
As it stands PDF.js works in Chrome but there’s no extension/plugin for it yet. Someone get on that – I want.