I could only find a reference about small arms of a small 5cm mortar crew in Heer (source:
www.ostfront.ru, Russian reconstructors of German WW2 army, website available in Russian/English/German). It consisted of 3 members: commander (rifle), cremember #1 (pistol), crewmember #2 (pistol).
Assuming that the crew commander wasn't actually using the mortar himself, he was effectively performing the functions of the scout class currently ingame, which indeed has a rifle.
I would also assume that, since 81 mm mortar was considered "heavy" (until many captured 120 mm ones were introduced) and had a higher typical engagement range, its crew of 6 men most likely had a maximum of 1 rifle for the commander. However, I couldn't find a source which would explicitly say that.
On the other hand, when the Russians are in game, I would prefer to have them authentically having rifles, since every crewmember even for a 120 mm mortar was issued a rifle (whether thay actually carried it with them all the time remains a question) and the commander was sometimes an SMG later in the war.
I didn't find anything on the British and US mortar crews, but I wasn't even looking