In a large saucepan, combine water, butter, and salt. Bring to a boil over medium heat.
Add the flour all at once. Stir until a smooth ball forms. Remove from heat. Let stand for 5 minutes.
Preheat oven to 400°F. Line a baking sheet with parchment paper. Spray with nonstick spray.
Add eggs, one at a time, beating well after each addition.
Continue beating until mixture is smooth and shiny.
Drop by ¼ cupfuls onto 2 baking sheets, spacing them 5" apart. Use a second baking sheet, if necessary, to maintain adequate spacing.
Combine milk and egg yolk. Stir to mix. Brush the puffs with this mixture.
Bake at 400°F until puffs rise and turn a light golden brown, about 30-35 minutes.
Remove to wire racks to cool.