openat(AT_FDCWD, "/dev/dsp", O_RDWR) = -1 EINVAL (Invalid argument)

[Randrianasulu]

I was trying to see why old proprietary application was failing to play sound with osspd (self compiled on Slackware 15.0 i586 with libfuse3 3.16.2 recompiled from -current).

It turned out it was using openat with O_RDWR argument, while sox/mplayer were using O_WRONLY and thus were unaffected by this bug.

Actually, tests seems to fail for ro case too:

guest@slax:~/botva/src/src/ossp$ ./osstest                                                         test_open@120 test succeeded (mixerfd >= 0)
test_open@129 test failed (ro_fd >= 0): Invalid argument
test_open@137 test succeeded (wo_fd >= 0)
test_wo@81 test succeeded (ret < 0)
test_wo@84 test succeeded (ret >= 0)
test_wo@87 test succeeded (ret < 0)
test_wo@89 test succeeded (errno == EINVAL)
test_wo@92 test succeeded (ret >= 0)
test_open@145 test failed (rw_fd >= 0): Invalid argument
test_mixer@160 test succeeded (ret >= 0)
Mixer id: OSS Proxy
Name: Mixer
Tests: 2 errors 8 success

git commit 72f82cc

I enabled alsa provider because I do not use pulseaudio normally.

I also set

/etc/modprobe.d/soundcore.conf
with single line
options soundcore preclaim_oss=0

proprietary application in question:
https://web.archive.org/web/20060615000000*/http://www.linuxmedialabs.com/LMLCD/contrib/RPMS/MainActor-3.6-5.i386.rpm

extract into /opt (I used midnight commander)

You can see what it tries to do with /dev/dsp by using
strace -P /dev/dsp app

kernel level OSS emulation with snd-pcm-oss module as itself works, aoss not.

Not sure from where this EINVAL comes? I tried running app and tests with sudo just in case, but they fail like for normal user.

you can try to compile dsp_info program from this page:

https://www.gbppr.net/guerrilla.net/reference/dsp/prog_dsp.htm

it shows similar error by default but if you set flag for openat to O_WRONLY it works.

[davidebeatrici]

O_RDWR means input + output, while O_WRONLY means output only:

ossp/ossp-alsap.c

Lines 216 to 228 in 72f82cc

	if (access == O_WRONLY || access == O_RDWR) {
	ret = snd_pcm_open(&pcm[PLAY], "default",
	SND_PCM_STREAM_PLAYBACK, block);
	if (ret >= 0)
	ret = set_hw_params(pcm[PLAY]);
	}
	if (ret >= 0 && (access == O_RDONLY || access == O_RDWR)) {
	ret = snd_pcm_open(&pcm[REC], "default",
	SND_PCM_STREAM_CAPTURE, block);
	if (ret >= 0)
	ret = set_hw_params(pcm[REC]);
	}

Does your system perhaps only have output devices?

[Randrianasulu]

O_RDWR means input + output, while O_WRONLY means output only:

ossp/ossp-alsap.c

Lines 216 to 228 in 72f82cc

	if (access == O_WRONLY || access == O_RDWR) {
	ret = snd_pcm_open(&pcm[PLAY], "default",
	SND_PCM_STREAM_PLAYBACK, block);
	if (ret >= 0)
	ret = set_hw_params(pcm[PLAY]);
	}
	if (ret >= 0 && (access == O_RDONLY || access == O_RDWR)) {
	ret = snd_pcm_open(&pcm[REC], "default",
	SND_PCM_STREAM_CAPTURE, block);
	if (ret >= 0)
	ret = set_hw_params(pcm[REC]);
	}

Does your system perhaps only have output devices?

I do not think so? This is tandart desktop motherboard with integrated sound and hdmi (unused) hanging from gt710 gpu ...

guest@slax:/dev/shm$ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
lavrate
    Rate Converter Plugin Using Libav/FFmpeg Library
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler                                                    oss
    Open Sound System
pulse
    PulseAudio Sound Server
speex
    Plugin using Speex DSP (resample, agc, denoise, echo, dereverb)
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
sysdefault:CARD=SB                                                                                     HDA ATI SB, ALC887-VD Analog
    Default Audio Device
front:CARD=SB,DEV=0
    HDA ATI SB, ALC887-VD Analog                                                                       Front output / input                                                                           usbstream:CARD=SB                                                                                      HDA ATI SB                                                                                         USB Stream Output                                                                              usbstream:CARD=NVidia                                                                                  HDA NVidia                                                                                         USB Stream Output                                                                              sysdefault:CARD=Loopback                                                                               Loopback, Loopback PCM
    Default Audio Device
front:CARD=Loopback,DEV=0
    Loopback, Loopback PCM
    Front output / input
usbstream:CARD=Loopback
    Loopback
    USB Stream Output
guest@slax:/dev/shm$

While I have aloop setup for recording outgoing sound, it may interfere .....

[Randrianasulu]

so, making capture device name "sysdefault" instead of "default" fixes "Device busy" err in MainActor but osstest now tests more and errs more:

guest@slax:~/botva/src/src/ossp$ ./osstest                                                         test_open@120 test succeeded (mixerfd >= 0)                                                        test_open@129 test succeeded (ro_fd >= 0)                                                          test_ro@59 test succeeded (ret >= 0)                                                               test_ro@62 test succeeded (ret < 0)
test_ro@65 test succeeded (ret >= 0)                                                               test_ro@68 test succeeded (ret < 0)
test_ro@70 test succeeded (errno == EINVAL)
test_open@137 test succeeded (wo_fd >= 0)
test_wo@81 test succeeded (ret < 0)
test_wo@84 test succeeded (ret >= 0)
test_wo@87 test succeeded (ret < 0)
test_wo@89 test succeeded (errno == EINVAL)
test_wo@92 test succeeded (ret >= 0)
test_open@145 test succeeded (rw_fd >= 0)
test_rw@103 test succeeded (ret >= 0)
test_rw@106 test succeeded (ret >= 0)
test_rw@109 test succeeded (ret >= 0)
test_rw@112 test succeeded (ret >= 0)
test_mixer@160 test succeeded (ret >= 0)                                                           Mixer id: OSS Proxy
Name: Mixer
test_trigger@172 test succeeded (ret == 0)
test_trigger@173 test succeeded (i == (PCM_ENABLE_INPUT|PCM_ENABLE_OUTPUT))
test_trigger@177 test succeeded (ret == 0)
test_trigger@178 test succeeded (i == 0)
test_trigger@182 test succeeded (ret == 0)
test_trigger@183 test succeeded (i == (PCM_ENABLE_INPUT|PCM_ENABLE_OUTPUT))
test_trigger@186 test succeeded (ret == 0)
test_notify@221 test failed (0): Fragsize: 170, bytes: 1360

test_notify@225 test succeeded (ret == bi.fragsize)
test_notify@225 test succeeded (ret == bi.fragsize)
test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@225 test succeeded (ret == bi.fragsize)                                                test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                                                                                                                        test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                     
test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170
                                                                                                   test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                                                                                                                        test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                                                                                                                        test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                     
test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170
                                                                                                   test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                                                                                                                        test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                     
test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)                                                           test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170

test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170
                                                                                                   test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170                     
test_notify@234 test succeeded (ret > 0)
test_notify@239 test failed (ret == bi.fragsize): Returned: -11 instead of 170
                                                                                                   test_mmap@196 test failed (area != MAP_FAILED): Failed to map: No such device                                                                                                                         Tests: 22 errors 54 success                                                                        guest@slax:~/botva/src/src/ossp$

so I guess "fix" for such non-standart configs is to make alsa device configureble, may be via env variable?

[davidebeatrici]

Yes, absolutely. Making the ALSA device configurable would be ideal regardless of any errors that may appear when using the default one.

Out of curiosity, what happens if you replace default with pulse in the part of code I linked above?

[Randrianasulu]

I think I force-disabled "pulse" device for alsa (because it was hiding real muxer controls) by removing its config file ....

[Randrianasulu]

while I tested it with "pulse" and playback and tests still works like they did with "sysdefault". But I have not tested real recording functionality ....