fabric/copy

Copy from a source to a sink. Read the row count as steps.<id>.outputs.rowsCopied.

The step takes no nested steps:. It takes timeout: (HH:MM:SS or D.HH:MM:SS) and retry: { count, interval }; the default is the portal policy.

with.source

One of: postgres { connection, query | table { schema, table }, timeout }, table <dataset>, or raw <source JSON>. A query may embed ${{ sql(expr) }}.

with.sink

One of: csv <dataset> (plus quote-all, default true), table <dataset> (plus action: OverwriteSchema | Overwrite | Append), or raw <sink JSON>.

with.translator

"tabular" for the portal default type-conversion translator. Omitted by default.

with.staging

Enable staging. Default false.

Example

- id: load
  uses: fabric/copy
  timeout: 01:00:00
  retry: { count: 1, interval: 300 }
  with:
    source:
      postgres:
        connection: ${{ env.POSTGRES }}
        query: |-
          SELECT * FROM t WHERE d <= ${{ sql(vars.date) }}::date
    sink:
      csv: out_file

with, elsewhere in the file:

env:
  POSTGRES: 00000000-0000-0000-0000-000000000000
datasets:
  out_file:
    csv:
      lakehouse: brz_lakehouse
      folder: Sales/Orders
      file: out.csv