Mattering: Why say that nothing matters?

Typically, when I hear someone say "nothing matters," I am curious about why they say that. 

Many people would say that, at some level, nothing matters because we're all going to die, the sun will explode, and eventually everything we've ever known will be obliterated and turn into space dust. 

But...wait...

What if we weren't going to die and become space dust? What if we were going to live forever? Would anything matter? 

If we really never died, wouldn't everything matter even less? Wouldn't there be no reason to do anything today versus tomorrow, because we have forever? 

Plus, how would/could we ever know that anything we did in our infinite lives had any meaning? 

So, is there any hypothetical scenario in which things "matter" in this sense? I'm not sure.