Bike lanes can be very cheap to make, put down some of those large planters to forbid car traffic and voila :)

A city close to mine additional had a bike purchase voucher for lower income individuals to help subsidize bike purchases (which is a much bigger upfront investment compared to a bus ticket!)

Overall I think you bring some good points! Ultimately the best choice will depend heavily on the area and its context.

Really depends where, but why not both?