Use a blender or food processor to purée the bananas, milk, peanut butter, sugar, and vanilla.
Sprinkle 1 tablespoon chopped roasted peanuts into plastic molds or small paper cups before adding the banana mixture. Divide the mixture evenly among the molds or cups.
Freeze for about 45 minutes, then insert a wooden stick into each; the pops will have frozen enough that the stick stays upright. Freeze for another 2 hours or until solid.
Run the molds or cups under cool running water for a few seconds to loosen the pops, then enjoy.