Organisers of the 2032 Olympic Games in Brisbane have expressed their fears over the proposed cost of the event.