Preheat oven to 350 degrees. Line muffin tins with paper liners and set aside.
In a large bowl, cream butter and sugar together until light and fluffy, about 4 minutes. Add eggs in 1 at a time until each is incorporated. Then add in vanilla, vanilla beans, sour cream and milk. Scrape sides and then mix again briefly.
In a separate bowl, whisk together all dry ingredients.
Add in dry ingredients little by little until everything is just incorporated. Mix the batter by hand using a spatula to ensure it is completely mixed.
Scoop batter into each cup so it's 3/4 full. Bake 25-30 minutes or until they are very lightly golden on top and toothpick comes out clean when inserted. Set aside to cool completely.
For the frosting, mix butter and powdered sugar together on low speed. Gradually add in vanilla and just enough milk to make frosting smooth and spreadable. Whip 3 minutes to make it light and fluffy.
*I use dried vanilla beans from Williams Sonoma. Feel free to substitute vanilla bean paste or the seeds scraped from a vanilla bean.
**If you want to pipe your frosting like in the pictures, double the frosting recipe. If you're using a knife to spread the frosting, the recipe listed will be enough.
Recipe based off of this muffin recipe from Ina Garten with a few minor changes.