The insulin signaling pathway is critical in regulating glucose levels and is associated with diabetes, obesity and longevity. An initial tyrosine phosphorylation cascade creates docking sites for protein interactions, initiating subsequent propagation of the signal throughout the cell. The phosphotyrosine interactome of this medically important pathway has not yet been studied comprehensively. We therefore applied quantitative interaction proteomics to exhaustively profile all potential phosphotyrosine (pTyr)-dependent interaction sites in its key players. We targeted and compared insulin receptor substrates 1 and 2 (IRS-1 and IRS-2) as central distributors of the insulin signal, the insulin receptor (InsR), the insulin-like growth factor 1 receptor (IGF1R), and the insulin receptor related receptor (IRR). Using the SILAC approach with phosphorylated vs. non-phosphorylated bait peptides, we found phosphorylation-specific interaction partners for 52 out of 109 investigated sites. In addition, doubly- and triply phosphorylated motifs provided insight into the combinatorial effects of phosphorylation events in close proximity to each other. Our results retrieve known interactions and substantially broaden the spectrum of potential interaction partners of IRS-1 and IRS-2. A large number of common interactors rationalize their extensive functional redundancy. However, several proteins involved in signaling and metabolism interact differentially with IRS-1 and IRS-2 and thus provide leads into their different physiological roles. Differences at the receptor level are reflected in multisite recruitment of SHP2 by the IGF1R and limited interactions with the IRR. In common with other recent reports, our data furthermore hint at non-SH2 or PTB domain mediated binding.