pyspark.sql.functions.lit() not nullable conversion

Question

As I create a new column with F.lit(1), while calling printSchema() I get

column_name: integer (nullable = false)

as lit function docs is quite scarce, do you think there is any simple mapping that can be done to turn it into nullable = true?

As explained [here](https://stackoverflow.com/questions/68578277/adding-a-nullable-column-in-spark-dataframe), instead of `F.lit(1)`, you can use `F.when(F.lit(True), F.lit(1))` — ZygD, May 07 '22 at 11:53

score 0 · Answer 1 · answered May 06 '22 at 14:35

0

Okay, in this scenario (only some specific column mapping, nothing in bulk) it seems like

df.schema['column_name'].nullable = True

does the trick. Nevertheless df.printSchema() isn't updated, although df.schema is.

answered May 06 '22 at 14:35

mckraqs

Hey, Did you figure out why printSchema and schema are giving different result? – Python Learner May 10 '22 at 10:32

1 Answers1