I feel I wasn't clear enough. Think about spells in terms of tab A slot B.
"A" is your word, gesture, thought, whatever - it's always with you. "B" is everything else. It takes a lot of time and effort to create the B part. Those come either from you spending months of off-screen studies or by a scroll scriber who makes a living out of it.
So when you prepare a spell, you sort of copy the B part into your mind for the remaining day and voila: the spell's at your disposal. When you cast from a scroll you just use someone else's effort.
Logically, if a player wants to use their spellbook page as a B-part blueprint, I see no reason to turn them down, but later they won't be able to just write it back in a minute, because they only know the A part.