Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LAPACK tests for OpenBLAS not controlled by maxparallel #3441

Open
dithwick opened this issue Sep 11, 2024 · 0 comments
Open

LAPACK tests for OpenBLAS not controlled by maxparallel #3441

dithwick opened this issue Sep 11, 2024 · 0 comments
Milestone

Comments

@dithwick
Copy link
Contributor

Hi,

While building OpenBLAS-0.3.24-GCC-13.2.0.eb on an AMD 7742 (with hyperthreading enabled so 256 logical cores), I noticed that the lapack tests were taking a very long time to run (more than a day) because the tests were starting a thread per logical core. For example, with the xeigtsts test I was observing:

$ ps -ef | grep 145558
username  145558  145557 99 16:31 pts/3    00:14:37 /dev/shm/username/build/OpenBLAS/0.3.24/GCC-13.2.0/OpenBLAS-0.3.24/lapack-netlib/TESTING/EIG/xeigtsts

$ ps -o nlwp 145558
NLWP
 256

I'm aware of the changes to the framework to set a default for maxparallel (in easybuilders/easybuild-framework#4606) but setting this manually in the easyconfig file did not control the number of threads used for the test. In the end, adding the following to the easyconfig worked:

pretestopts = 'export OMP_NUM_THREADS=16 &&'

After running the build again, this took about 5 minutes to complete the tests instead of many hours.

Rather than adding this to all of the easyconfig files, I/we were wondering if there is a way of adding this to the easyblock so there's a default and/or controlled by maxparallel by editing the line

cmd = "%s make %s %s" % (self.cfg['pretestopts'], runtest, self.cfg['testopts'])
somehow (see the conversation on slack)?

Thanks

@boegel boegel added this to the 4.x milestone Sep 25, 2024
@boegel boegel changed the title lapack tests for OpenBLAS not controlled by maxparallel LAPACK tests for OpenBLAS not controlled by maxparallel Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants