I guess you will just have to make small talk with them, the weather...compliment them on small stuff...then when grandson is in the car you can ask where he would like to eat, etc. then when dropping him off, just say you enjoyed your time with GS. Let him decide what he wants to share.
For sure, ask his parents about specific activities and permission. But just try not to give the grandparents any ammunition.
Ksm