I wouldn't have thought it would be a soap residue, as you would know about it when using them, when I had that with mine they developed a bed ammonia smell when wee'd in. Perhaps try a wash with apple cider vinegar in the fabric softer drawer?
If you live in a hard water area then 1tsp of powder is nowhere near enough. I would probably also try washing at a higher temp I think. I use a full dose of bio powder. (usually Bold 2 in 1 as that helps keep them softer) I cold rinse, wash at 40 (sometimes at 60), then 2/3 rinses at the end.