In a medium saucepan, whisk together the brown sugar, cornstarch, and salt. Gradually add the milk while whisking to combine.
Cook the mixture over medium heat, stirring constantly until it thickens and begins to bubble, about 8-10 minutes.
Once thickened, remove the saucepan from the heat. Slowly stir a small amount of the hot mixture into the beaten egg yolks to temper them, then return the egg yolk mixture to the saucepan.
Cook for an additional 2 minutes, stirring constantly. Remove from heat and stir in the butter and vanilla extract until fully combined.
Pour the butterscotch filling into the pre-baked pie crust, smoothing the top with a spatula.
Allow the pie to cool at room temperature for about 30 minutes, then refrigerate for at least 2 hours to set.
Before serving, top with whipped cream if desired.