Cleaning vintage bottles is a task that blends history ... Baking Soda Paste: Mix baking soda with water to form a paste and apply it with a soft cloth or brush to stained areas.