Skip to content

Ex2 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
darkushin wants to merge 62 commits into
base: master
Choose a base branch
from
Open

Ex2 #2

darkushin wants to merge 62 commits into from

Conversation

@darkushin
Copy link
Owner

@darkushin darkushin commented May 3, 2021

In this part, we will handle the data piece-wise, not assuming the entire data fits into the main memory.

darkushin and others added 30 commits May 3, 2021 15:16
@darkushin
Ex2 first commit
b24ee90
@nirnits
DataLoader added.
aa21bcc
@darkushin
updated data parser
aa14759
@nirnits
dl change
d679760
@darkushin
addReviewText before saving files
be529b3
@nirnits
sortBuffer + saveBuffer
c9b9b66
@darkushin
Before sorting tokens alphanumerically
c3c382e
@darkushin
Added ExternalMergeSort - before implementing
3eb81ec
@darkushin
ExternalMergeSort
bfdfa1c
@darkushin
ExternalMergeSort - before debugging
dc57512
@darkushin
ExternalMergeSort - train debugging
c66a861
@darkushin
Beginning of TokenIndex
ed15c2b
@darkushin
prepareTokenDict
3cbe607
@nirnits
other dicts
12d27f2
@nirnits
Sorting works?
f681df9
@nirnits
EM sort should be working (almost?)
87b6bd5
@nirnits
sort improvements
1b19874
@nirnits
Token Index is (almost?) done.
baad054
@darkushin
Code after debugging, before cleaning
3a5e5dc
@darkushin
Cleaned code, before optimization
da01cbe
@darkushin
Updated dictString writing in TokensIndex
d8c4a18
test cs computers
5e6ebbe
@nirnits
time
f427cdb
@nirnits
Merge branch 'Ex2' of https://github.com/darkushin/Web-Infromation-Re…
53040fc 
…trieval into Ex2
@darkushin
Added time info
4ca2e6d
@nirnits
Merge branch 'Ex2' of https://github.com/darkushin/Web-Infromation-Re…
8912132 
…trieval into Ex2
@darkushin
saveInvertedIndex time improvement
4281252
@nirnits
Merge branch 'Ex2' of https://github.com/darkushin/Web-Infromation-Re…
bde646d 
…trieval into Ex2
@darkushin
Before changing saveInvertedIndex to save integers
1981b1e
@darkushin
Buffered RandomAccessFile
9e035c3
nirnits and others added 30 commits May 25, 2021 12:26
@nirnits
Merge branch 'Ex2' of https://github.com/darkushin/Web-Infromation-Re…
e2beb22 
…trieval into Ex2
@darkushin
Changed reviewIndex to use hashMap
4377a53
@darkushin
Improved tokenIndex, before changing all LinkedLists
e405f22
@darkushin
Removed LinkedLists
95aafb6
@darkushin
TokensIndex without encoding
b10d7a9
@nirnits
Merge branch 'Ex2' of https://github.com/darkushin/Web-Infromation-Re…
52b165a 
…trieval into Ex2
@darkushin
Basic Test are working. before investigating data loader & parser
c85587d
@nirnits
groupvarint encoding for inverted index
5c80f65
@nirnits
encoding merge
1a45ac9
@nirnits
changed stringInfo to int
2f70bc3
@darkushin
Converting short->int
8339097
@darkushin
External Merge Sort logs
4137867
@darkushin
Analysis code
e3535cb
@darkushin
Text Creator tests
9c3fd41
@nirnits
DataParser + Loader changes
ee93588
@nirnits
again
6a77a77
@darkushin
Updated Data Parsing
6f44fcc
@darkushin
Cleaned Code, with time measuring
d0e2e3c
@darkushin
Changed switch-case in DataParser
19e6879
@darkushin
Cleaned code for submission, before running tests again
7287e07
@darkushin
Submission code
c12c018
@darkushin
Final Submission
42ea768
Ex2 fix
6ffc156
@nirnits
fixed reviewdict bug
b8cc081
@nirnits
commented memory improvements
42089ee
@darkushin
not working changes
c00346e
@nirnits
reviewindex save/load
6e384a6
@nirnits
reviewIndex save/load #2
3728348
@darkushin
working code - before cleanup
b43ef58
@darkushin
Ex2 - fixed code
5baa5b4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@darkushin @nirnits