Is this his warning sign?

Yes, I think it is a warning sign. And I have to agree with him, even though you are busy, you should take at least 10 minutes out of your day to give him a call - not a text or an email, but a phone call which is more personal. And will let him know that you are truly thinking of him, even though you don't have a lot of time to spend with him.