Across the board, Australians are reading less, with young men reading the least and older women reading the most.