Albanese has spoken out after the government was accused of breaking its election promises around negative gearing.