Normally parting with physical memories makes people sad. Some people hold on to memories forever because sometimes only memories remain while other things get lost over time.
Then you ask, "Can you be sad losing something that didn't belong to you?"
Nope. I mean, how can you lose something;
if you didn't own it from the start?