# Upload pkl predictions

**URL:** <https://forum.numer.ai/t/upload-pkl-predictions/7005>\
**Category:** Tournament\
**Created:** [January 31, 2024, 2:50pm UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005 "2024-01-31T14:50:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![marc\_gg](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/marc_gg/32/2855_2.png) [@marc\_gg](https://forum.numer.ai/u/marc_gg)\
**Post date:** [January 31, 2024, 2:50pm UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005/1 "2024-01-31T14:50:26Z")

</div>

Hi people!

I am trying to upload a prediction function using cloudpickle. It works fine if all the code is in the same file, but when creating a folder structure, the pkl is uploaded but it can not be extracted correctly as it miss modules. If the main file is like:

from my\_module import my\_func  
…  
def predict(live\_features: pd.DataFrame) → pd.DataFrame:  
features = my\_func(live\_features)  
live\_predictions = model.predict(lfeatures)  
submission = pd.Series(live\_predictions, index=live\_features.index)  
return submission.to\_frame(“prediction”)

In this case how I should proceed? Numerai tells me “my\_module” is not found

Thanks in advance

---

<div class="post-metadata">

**Author:** ![shatteredx](https://avatars.discourse-cdn.com/v4/letter/s/ee59a6/32.png) [@shatteredx](https://forum.numer.ai/u/shatteredx)\
**Post date:** [January 31, 2024, 8:53pm UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005/2 "2024-01-31T20:53:51Z")

</div>

I think you have to keep all the code in the same file. Maybe someone can prove me wrong?

---

<div class="post-metadata">

**Author:** ![rpica](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/rpica/32/2854_2.png) [@rpica](https://forum.numer.ai/u/rpica)\
**Post date:** [April 18, 2024, 7:29am UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005/3 "2024-04-18T07:29:54Z")

</div>

This is really frustrating 😔, I think it’s the first time I’m punished for giving code some structure…

Did someone find a solution or we have to copy paste all related code into a single python module to make cloudpickle work? (and hence, model uploads, which is an amazing feature)

---

<div class="post-metadata">

**Author:** ![marc\_gg](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/marc_gg/32/2855_2.png) [@marc\_gg](https://forum.numer.ai/u/marc_gg)\
**Post date:** [April 18, 2024, 10:54am UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005/4 "2024-04-18T10:54:58Z")

</div>

Completly agree, it is frustrating. It did not manage to solve the issue, I finally decided to make the training, evaluation and some other stuff with a code using structure, but when submitting I have a single code that loads the models and the feature list, the only issue I have to copy-paste the same code to recreate the generated-variables, as importing the code does not work, but importing list and models it does.

---

<div class="post-metadata">

**Author:** ![0xleo](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/0xleo/32/3110_2.png) [@0xleo](https://forum.numer.ai/u/0xleo)\
**Post date:** [April 20, 2024, 4:10pm UTC](https://forum.numer.ai/t/upload-pkl-predictions/7005/5 "2024-04-20T16:10:30Z")

</div>

It’s marked as experimental, but have you tried marking your module to be picked by value instead of by reference?

> **[GitHub - cloudpipe/cloudpickle: Extended pickling support for Python objects](https://github.com/cloudpipe/cloudpickle#overriding-pickles-serialization-mechanism-for-importable-constructs)**
>
> Extended pickling support for Python objects

```auto
import cloudpickle
import my_module
cloudpickle.register_pickle_by_value(my_module)
cloudpickle.dumps(my_module.my_function) # my_function is pickled by value
cloudpickle.unregister_pickle_by_value(my_module)
cloudpickle.dumps(my_module.my_function) # my_function is pickled by reference

```