In the beginning they kept the name: it was called Provincia Judaea.
Later they changed the name to Palaestina (after a failed Jewish rebellions when Romans wanted to retaliate and make their rule more visible. The new name came from the enemies of the Jews, Philistines) and the province was called "Provinciae Syria-Palaestina".
