If I was to fix this, I'd do it much like with fiberglass, but with wood glue, sawdust, heavy paper, and solid wood.
I'd mix Titebond III and coarse sawdust to make a paste to fill in the big holes. The outside vinyl should hold it, but I'd make sure that doesn't bulge or get lumpy. Then, I'd...