There is nothing wrong with cleaning the nest box. The babies will hiss and the parents will leave but they will come back. I take the clutch of babies out and put them in a plastic bowl, scrape down the sides of the box (I use a wide plastic paint scraper for this)and empty out the litter. I fill the box with clean litter and replace the babies. The parents will come back. But A worm in the box? I havent had that type of problem before. Are they bred out side? If not, are you sure it was a worm and not a fly maggot? I breed my birds in a 10 by 12 foot shed and since insects get in I use fly strip paper to control any stray flies. I personally would change the box, but thats me. Other breeders on here might tell you not to. But an egg's shell is porous. If the previous chick died of some disease and the litter is harboring some bacteria it might transfer to the new eggs. I would do the same thing I described above but work faster since it sounds as if they are sitting tight. You will want to get the egg back in as quickly as possible so it doesn't get cold for long. Gently take the egg out put it somewhere safe and clean the box. Wait til both birds are out of the cage. if possible. If not I would do it when the female is on the egg. In my experience the mother is not as nasty as the father when it comes to protecting the egg Just do not shake the egg. you dont want to addle the embryo. I am not trying to overly scare you. Just point out to be careful. If you can post a pic of the nest box that would be good too. I have never heard of one that doesnt not have a door you can check on the eggs. Unless it is home made. I would not change the box. A change like that may cause them to abandon the eggs.