Skip to content

Reduce memory usage #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
hmgaudecker merged 3 commits into from
Jul 16, 2025
Merged

Reduce memory usage #82

hmgaudecker merged 3 commits into from
Jul 16, 2025

Conversation

@mj023
Copy link
Collaborator

@mj023 mj023 commented Apr 7, 2025 •
edited
Loading

This PR introduces multiple measures to reduce the memory usage of the gradient computation in skillmodels.

  • Move rematerialization checkpoint to the scans body function
  • Introduce the option to split the dataset and then add up the gradient values to save memory
  • Improve householder product with faster matrix updating

@mj023 mj023 requested a review from hmgaudecker April 27, 2025 15:46
hmgaudecker
hmgaudecker approved these changes Jul 16, 2025
View reviewed changes
Copy link
Member

@hmgaudecker hmgaudecker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for taking so long to look at this. Very, very nice -- thanks a lot!!!

@hmgaudecker hmgaudecker merged commit bbe69ce into endogenize_investments Jul 16, 2025
0 of 3 checks passed
@hmgaudecker hmgaudecker deleted the reduce_memory branch July 16, 2025 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hmgaudecker hmgaudecker hmgaudecker approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mj023 @hmgaudecker