Plasmonic biosensors are increasingly being used for the analysis of extracellular vesicles (EVs) originating from disease areas. However, the high non-specific binding of EVs to a gold-sensing surface has been a critical problem and hindered the true translational potential. Here, we report that direct antibody immobilization on the plasmonic gold surface via physisorption shows excellent capture of cancer-derived EVs with ultralow non-specific binding even at very high concentrations. Contrary to commonly used methods that involve thiol-based linker attachment and an EDC/sulfo-NHS reaction, we show a higher specific capture rate and >50-fold lower non-specific on citrate-capped plain and nanopatterned gold surfaces. The method provides a simple, fast, and reproducible means to functionalize plasmonic gold surfaces with antibodies for robust EV biosensing.