Comment by giancarlostoro

Comment by giancarlostoro 3 days ago

8 replies

View on Hacker News

Lowkey I hate the "\" line continuation in Python to force PEP-8 compliance in a way... Is there any Pythonistas who would write the examples in there differently to achieve a similar level of readability?

> pipeline = task(get_data, branch=True) \

> | task(step1, workers=20) \

> | task(step2, workers=20) \

> | task(step3, workers=20, multiprocess=True)

dec0dedab0de 3 days ago

I really don’t like overloading pipes like this. I would rather chain methods like how the django orm does it.

you could reassign every line, but it would look nicer with chained functions.

  pipeline = task(get_data, branch=True)
  pipeline = pipeline | task(step1, workers=20)
  pipeline = pipeline |  task(step2, workers=20)
  pipeline = pipeline |  task(step3, workers=20, multiprocess=True)
edit:

I would be tempted to do something like this:

  steps = [task(step1, workers=20),
           task(step2, workers=20),
           task(step3, workers=20, multiprocess=True)]
  pipeline = task(get_data, branch=True)

  for step in steps:
      pipeline =   pipeline.__or__(step)
Reply View 2 replies
  • Rickster35 3 days ago

    According to the docs, | is syntactic sugar for the .pipe method.

      pipeline = task(get_data, branch=True).pipe(
      task(step1, workers=20)).pipe(
      task(step2, workers=20)).pipe(
      task(step3, workers=20, multiprocess=True))
    That's probably the chained method approach for those with this preference.
    Reply View 0 replies
  • me-vs-cat a day ago

    This style looks pretty good to me:

        pipeline = task(...)
    pipeline |= task(...)
    So does this style:

        steps = [task(...), task(...)]
    pipeline = functools.reduce(operator.or_, steps)
    But it appears you can just change "task" to "Task" and then:

        pipeline = pyper.Pipeline([Task(...), Task(...)])
    Reply View 0 replies
morkalork 3 days ago

Wrap the statement in (...) and you can drop the backslashes. See also how people split up complicated loc queries in pandas on multiple lines too.

Reply View 4 replies
  • giancarlostoro 3 days ago

    Thank you! It seems this works for not just parens but square brackets and curly braces too! Only special requirement is indentation is consistent (duh though).

    I've not been doing Python day-to-day so I'm starting to lose my touch on all the nice little tricks.

    Reply View 3 replies
    • Rickster35 3 days ago

      Yeah, this seems neatest if you don't like line breaks

      pipeline = (

          task(get_data, branch=True)

    | task(step1, workers=20)

    | task(step2, workers=20)

    | task(step3, workers=20, multiprocess=True)
      )

      Square brackets would create a list and braces would create a set of course. The contents still can be split over different lines-- just pointing that this syntax doesn't do the same thing.

      Reply View 2 replies