Get integration tests running in CI

[jaqx0r]

@3c2b2ff5 contrinubted a samba integration test script but it needs a bit of work to run in Travis CI. I'd also like to turn my (not submitted) OpenLDAP test script into an integration test.

So two goals:

• make the integration tests runnable manually
• incorporate the tests on CI. We don't have to keep Travis if, say, spinning up a docker container on Circle to run the source server is significantly easier.

[3c2b2ff5]

I have setup a new branch CircleCi without travis.
I noticed that when ahving python3-ldap in requirements.txt the build will fail, with python-ldap the build succeeds, even when running the tests with Python 3.
Next step will be to spinn up a docker container with samba 4 AD and another one with openldap or slapd.
I still need to figure out how to test against the source server.

[jaqx0r]

Awesome. The python package's name is `python-ldap`, the Debian package calls it `python-ldap` or `python3-ldap`. So don't put python3-ldap in requirements.txt. How are you testing your Circle config?

…

On Sat, 2 Nov 2019 at 05:43, 3c2b2ff5 ***@***.***> wrote: I have setup a new branch CircleCi <https://github.com/3c2b2ff5/nsscache/tree/CircleCi> without travis. I noticed that when ahving python3-ldap in requirements.txt the build will fail, with python-ldap the build succeeds, even when running the tests with Python 3. Next step will be to spinn up a docker container with samba 4 AD and another one with openldap or slapd. I still need to figure out how to test against the source server. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX62JPQJAV5OQYCIXCPDQRR2G3A5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC32GAA#issuecomment-548905728>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX65XFGRPJIVANO4YWQLQRR2G3ANCNFSM4JHUUQDQ> .

[3c2b2ff5]

currently I created a circleci account and it runs successfully, with the difference to travisci that you cannot choose a branch, at least I couldn't finde it.
It seems samba cannot be installed because of the file system circleci is using:

ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.

But it is really fast. The CI runs in less than a minute.
There is a possibility to run circleci local ci, look here.

[3c2b2ff5]

Now the build in CircleCi runs successful, samba is installed and the domain provisioned, the users, the groups and the group members are are successfully added, there are still some issues to be solved.

Samba is not being run as an AD Domain Controller: Masking samba-ad-dc.service

ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.

And a lot of:

Could not find machine account in secrets database: Failed to fetch machine account password for LOCAL from both secrets.ldb (Could not find entry to match filter: '(&(flatname=LOCAL)(objectclass=primaryDomain))' base: 'cn=Primary Domains': No such object: dsdb_search at ../source4/dsdb/common/util.c:4705) and from /var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO

[3c2b2ff5]

I am tending now to run CircleCi loacally or on CircleCi just to test the config and to run python3 runtests.py. Then spinn up two docker containers locally, one with nsscache and the other one with the source server.
The nsscache container should run nsscache installation with a modified nssccache.conf to contact openldap | AD running on the other container.
I think by creating a docker network and connecting both containers to it, they should be able to communicate. Of course we can run nsscache and openldap | AD on the same container.
An ldap query to AD must be encrypted. For the test the client just needs the default certificate form the AD, which is created with the installation:

# echo -n | openssl s_client -connect ad.host:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /usr/local/share/ca-certificates/ad.crt
# update-ca-certificates

Do you have further ideas?

[3c2b2ff5]

I have managed to install samba4 AD with Travis. nsscache -d update -f fails because of permissions, it cannot lock /var/run/nsscache. nsscache -d verify has always 3 warnings, but no errors.
Running both commands with sudo didn't work neither, I think because of the environment.
The new branch is here.
I hope you can fix it.

[3c2b2ff5]

I fixed the verify command by adding a nsswitch.conf file. The update still fails because of the lacking permissions to lock /var/run/nsscache.

[3c2b2ff5]

I fixed it in the last commit. I had to switch the verify and the update commands, otherwise when running the verify at the end, it gives the error that the cache entries in passwd/group/shadow are not available via NSS.

[3c2b2ff5]

Any ideas why running verify after update result in the mentioned error?

[jaqx0r]

I haven't had a chance to test yet, but I'll investigate layer today or tomorrow.

…

On Fri, 8 Nov 2019, 13:32 3c2b2ff5, ***@***.***> wrote: Any ideas why running verify after update result in the mentioned error? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX6ZI7HPHDFVM3HQ2RBDQSVL7RA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDRH57I#issuecomment-551714557>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX6YCAAO24QDTDSBPYRDQSVL7RANCNFSM4JHUUQDQ> .

[3c2b2ff5]

cool. Thanks

[jaqx0r]

The CircleCI build now runs the OpenLDAP regression test as par tof the release. I discovered that the sambaSID changes broke this, so that's good that it caught them. I think making sambaSID an essential attribute only if use_rid is configured is the right thing to do.

I tried to get the samba4 script running, but I got stuck trying to figure out how to run samba as a non-root user.

[3c2b2ff5]

Indeed! Running openldap without samba3 schema will break the tests and nsscache. Normally you have the smaba3 schema integration within openldap. Using openldap without any extensions for authentication or as a directory service we need to modify the code at this point to satisfy a plain openldap installation.

To install and run samba4 AD you need to run the script samba.sh with sudo. Unfortunately there is no other way to do it. All the operations (installation/provisioning/user and group creation etc.) need root privileges.

[3c2b2ff5]

I cannot test now with CircleCi. I keep get the following error at the coveralls stage:

#!/bin/bash -eo pipefail
coveralls
/bin/bash: coveralls: command not found
Exited with code 127

I already created an account on coveralls. I have no prior experience with coveralls.