Add configurable Monte Carlo dropout uncertainty estimation #29
Conversation
ad12
left a comment
ad12
left a comment
There was a problem hiding this comment.
looking good! add some unittests as well to make sure things are working as you'd expect
| with h5py.File(save_name, "w") as h5f: | ||
| h5f.create_dataset("probs", data=output["y_pred"]) | ||
| h5f.create_dataset("labels", data=labels) | ||
| h5f.create_dataset("true", data=output["y_true"]) |
There was a problem hiding this comment.
i would avoid saving y_true - it should be easily accessible from your input data hdf5 file and duplicating it here would use up more disk space, which is limited.
| mc_dropout=False, | ||
| mc_dropout_T=100, |
There was a problem hiding this comment.
seems like these are not used - if that's the case, delete
|
|
||
| tmp_batch_outputs_mc_dropout = None | ||
| if mc_dropout: | ||
| tmp_batch_outputs_mc_dropout = np.stack([model(batch_x, training=True) for _ in range(mc_dropout_T)]) |
There was a problem hiding this comment.
I see what you're trying to do here, but it will not be reproducible, which is necessary if we are to add this in the inference loop. There is no random seed being set, so the features that are dropped out will be different if you run inference on the same example twice.
im not sure exactly how to account for this, potentially setting a random seed. Write a unittest to verify that this does in fact produce identical inputs when run twice.
| MC_DROPOUT = False | ||
| MC_DROPOUT_T = 100 |
There was a problem hiding this comment.
Add a comment indicating what MC_DROPOUT and MC_DROPOUT_T are referring to
2 participants
No description provided.