We show a general limit to performance of dispersive optical components based on a limit to the number of orthogonal waves that can be generated by an object of given composition. The limit applies generally to linear systems, including nanophotonic structures, and is illustrated with a one-dimensional example of dispersing pulses.