redpanda-data · mihaitodor · Sep 24, 2024
@@ -314,7 +314,7 @@
 ** xref:cookbooks:filtering.adoc[]
 ** xref:cookbooks:joining_streams.adoc[]
 ** xref:cookbooks:rag.adoc[]
-** xref:cookbooks:kafka_migrator.adoc[]
+** xref:cookbooks:redpanda_migrator.adoc[]
 
 
 
@@ -1,4 +1,4 @@
-= kafka_migrator
+= redpanda_migrator
 // tag::single-source[]
 :type: input
 :status: beta
@@ -9,11 +9,11 @@
 
 component_type_dropdown::[]
 
-Use this connector in conjunction with the xref:components:outputs/kafka_migrator.adoc[`kafka_migrator` output] to migrate topics between Apache Kafka brokers. The `kafka_migrator` input uses the https://github.com/twmb/franz-go[Franz Kafka client library^].
+Use this connector in conjunction with the xref:components:outputs/redpanda_migrator.adoc[`redpanda_migrator` output] to migrate topics between Apache Kafka brokers. The `redpanda_migrator` input uses the https://github.com/twmb/franz-go[Franz Kafka client library^].
 
 
 ifndef::env-cloud[]
-Introduced in version 4.35.0.
+Introduced in version 4.37.0.
 endif::[]
 
 [tabs]
@@ -26,7 +26,7 @@ Common::
 # Common configuration fields, showing default values
 input:
   label: ""
-  kafka_migrator:
+  redpanda_migrator:
     seed_brokers: [] # No default (required)
     topics: [] # No default (required)
     regexp_topics: false
@@ -43,7 +43,7 @@ Advanced::
 # All configuration fields, showing default values
 input:
   label: ""
-  kafka_migrator:
+  redpanda_migrator:
     seed_brokers: [] # No default (required)
     topics: [] # No default (required)
     regexp_topics: false
@@ -70,23 +70,23 @@ input:
       check: ""
       processors: [] # No default (optional)
     topic_lag_refresh_period: 5s
-    output_resource: kafka_migrator_output
+    output_resource: redpanda_migrator_output
 ```
 
 --
 ======
 
-The `kafka_migrator` input:
+The `redpanda_migrator` input:
 
 * Reads a batch of messages from a Kafka broker.
 * Attempts to create all selected topics along with their associated ACLs in the broker that the output points to, identified by the label specified in `output_resource`.
-* Waits for the `kafka_migrator` output to acknowledge the writes before updating the Kafka consumer group offset.
+* Waits for the `redpanda_migrator` output to acknowledge the writes before updating the Kafka consumer group offset.
 
 Specify a consumer group for this input to consume one or more topics and automatically balance the topic partitions across any other connected clients with the same consumer group. Otherwise, topics are consumed in their entirety or with explicit partitions.
 
 == Metrics
 
-This input emits a `input_kafka_migrator_lag` metric with `topic` and `partition` labels for each consumed topic.
+This input emits a `input_redpanda_migrator_lag` metric with `topic` and `partition` labels for each consumed topic.
 
 == Metadata
 
@@ -698,11 +698,11 @@ The period of time between each topic lag refresh cycle.
 
 === `output_resource`
 
-The label of the `kafka_migrator` output in which the currently selected topics need to be created before attempting to read messages.
+The label of the `redpanda_migrator` output in which the currently selected topics need to be created before attempting to read messages.
 
 
 *Type*: `string`
 
-*Default*: `"kafka_migrator_output"`
+*Default*: `"redpanda_migrator_output"`
 
 // end::single-source[]
@@ -1,4 +1,4 @@
-= kafka_migrator_bundle
+= redpanda_migrator_bundle
 // tag::single-source[]
 :type: input
 :status: experimental
@@ -10,24 +10,24 @@
 component_type_dropdown::[]
 
 
-The `kafka_migrator_bundle` input reads messages and schemas from an Apache Kafka or Redpanda cluster. Use this input in conjunction with the xref:components:outputs/kafka_migrator_bundle.adoc[`kafka_migrator_bundle` output].
+The `redpanda_migrator_bundle` input reads messages and schemas from an Apache Kafka or Redpanda cluster. Use this input in conjunction with the xref:components:outputs/redpanda_migrator_bundle.adoc[`redpanda_migrator_bundle` output].
 
 ```yml
 # Config fields, showing default values
 input:
   label: ""
-  kafka_migrator_bundle:
-    kafka_migrator: {} # No default (required)
+  redpanda_migrator_bundle:
+    redpanda_migrator: {} # No default (required)
     schema_registry: {} # No default (required)
     migrate_schemas_before_data: true
 ```
 
 
 == Fields
 
-=== `kafka_migrator`
+=== `redpanda_migrator`
 
-The xref:components:inputs/kafka_migrator.adoc[`kafka_migrator` input] configuration.
+The xref:components:inputs/redpanda_migrator.adoc[`redpanda_migrator` input] configuration.
 
 
 *Type*: `object`

@@ -1,4 +1,4 @@
-= kafka_migrator
+= redpanda_migrator
 // tag::single-source[]
 :type: output
 :status: beta
@@ -11,10 +11,10 @@ component_type_dropdown::[]
 
 Writes a batch of messages to an Apache Kafka broker and waits for acknowledgement before propagating them back to the input.
 
-Use this connector in conjunction with the xref:components:inputs/kakfa_migrator[`kafka_migrator` input] to migrate topics between Kafka brokers. The `kafka_migrator` output uses the the https://github.com/twmb/franz-go[Franz Kafka client library^].
+Use this connector in conjunction with the xref:components:inputs/kakfa_migrator[`redpanda_migrator` input] to migrate topics between Kafka brokers. The `redpanda_migrator` output uses the the https://github.com/twmb/franz-go[Franz Kafka client library^].
 
 ifndef::env-cloud[]
-Introduced in version 4.35.0.
+Introduced in version 4.37.0.
 endif::[]
 
 [tabs]
@@ -27,7 +27,7 @@ Common::
 # Common config fields, showing default values
 output:
   label: ""
-  kafka_migrator:
+  redpanda_migrator:
     seed_brokers: [] # No default (required)
     topic: "" # No default (required)
     key: "" # No default (optional)
@@ -52,7 +52,7 @@ Advanced::
 # All config fields, showing default values
 output:
   label: ""
-  kafka_migrator:
+  redpanda_migrator:
     seed_brokers: [] # No default (required)
     topic: "" # No default (required)
     key: "" # No default (optional)
@@ -84,18 +84,26 @@ output:
       client_certs: []
     sasl: [] # No default (optional)
     timestamp: ${! timestamp_unix() } # No default (optional)
-    input_resource: kafka_migrator_input
+    input_resource: redpanda_migrator_input
+    replication_factor_override: true
+    replication_factor: 3
 ```
 
 --
 ======
 
-This output can query the `kafka_migrator` input for topic and ACL configurations.
+This output can query the `redpanda_migrator` input for topic and ACL configurations.
 
 If the configured broker does not contain the current message topic, this output attempts to create it along with the topic
-ACLs, which are automatically read from the `kafka_migrator` input, identified by the label specified in
+ACLs, which are automatically read from the `redpanda_migrator` input, identified by the label specified in
 `input_resource`.
 
+ACL migration adheres to the following principles:
+
+- `ALLOW WRITE` ACLs for topics are not migrated
+- `ALLOW ALL` ACLs for topics are downgraded to `ALLOW READ`
+- Only topic ACLs are migrated, group ACLs are not migrated
+
 
 == Examples
 
@@ -109,14 +117,14 @@ Writes messages to the configured broker and creates topics and topic ACLs if th
 
 ```yaml
 output:
-  kafka_migrator:
+  redpanda_migrator:
     seed_brokers: [ "127.0.0.1:9093" ]
     topic: ${! metadata("kafka_topic").or(throw("missing kafka_topic metadata")) }
     key: ${! metadata("kafka_key") }
     partitioner: manual
     partition: ${! metadata("kafka_partition").or(throw("missing kafka_partition metadata")) }
     timestamp: ${! metadata("kafka_timestamp_unix").or(timestamp_unix()) }
-    input_resource: kafka_migrator_input
+    input_resource: redpanda_migrator_input
     max_in_flight: 1
 ```
 
@@ -808,11 +816,31 @@ timestamp: ${! metadata("kafka_timestamp_unix") }
 
 === `input_resource`
 
-The label of the `kafka_migrator` input from which to read the configurations of topics and ACLs for creation.
+The label of the `redpanda_migrator` input from which to read the configurations of topics and ACLs for creation.
 
 
 *Type*: `string`
 
-*Default*: `"kafka_migrator_input"`
+*Default*: `"redpanda_migrator_input"`
+
+
+=== `replication_factor_override`
+
+Use the specified replication factor when creating topics.
+
+
+*Type*: `bool`
+
+*Default*: `true`
+
+
+=== `replication_factor`
+
+Replication factor for created topics. This is only used when `replication_factor_override` is set to `true`.
+
+
+*Type*: `int`
+
+*Default*: `3`
 
 // end::single-source[]
@@ -1,4 +1,4 @@
-= kafka_migrator_bundle
+= redpanda_migrator_bundle
 // tag::single-source[]
 :type: output
 :status: experimental
@@ -10,23 +10,23 @@
 
 component_type_dropdown::[]
 
-Writes messages and schemas to an Apache Kafka or Redpanda cluster. Use this output in conjunction with the `kafka_migrator_bundle` input.
+Writes messages and schemas to an Apache Kafka or Redpanda cluster. Use this output in conjunction with the `redpanda_migrator_bundle` input.
 
 
 ```yml
 # Config fields, showing default values
 output:
   label: ""
-  kafka_migrator_bundle:
-    kafka_migrator: {} # No default (required)
+  redpanda_migrator_bundle:
+    redpanda_migrator: {} # No default (required)
     schema_registry: {} # No default (required)
 ```
 
 == Fields
 
-=== `kafka_migrator`
+=== `redpanda_migrator`
 
-The xref:components:outputs/kafka_migrator.adoc[`kafka_migrator` output] configuration.
+The xref:components:outputs/redpanda_migrator.adoc[`redpanda_migrator` output] configuration.
 
 *Type*: `object`
 

@@ -1,4 +1,4 @@
-= kafka_migrator_offsets
+= redpanda_migrator_offsets
 // tag::single-source[]
 :type: output
 :status: beta
@@ -11,12 +11,12 @@
 component_type_dropdown::[]
 
 
-Use the `kafka_migrator_offsets` output in conjunction with the `kafka_franz` input that is configured to read the `__consumer_offsets` topic.
+Use the `redpanda_migrator_offsets` output in conjunction with the `kafka_franz` input that is configured to read the `__consumer_offsets` topic.
 This output uses the https://github.com/twmb/franz-go[Franz Kafka client library^].
 
 
 ifndef::env-cloud[]
-Introduced in version 4.35.0.
+Introduced in version 4.37.0.
 endif::[]
 
 [tabs]
@@ -29,7 +29,7 @@ Common::
 # Common config fields, showing default values
 output:
   label: ""
-  kafka_migrator_offsets:
+  redpanda_migrator_offsets:
     seed_brokers: [] # No default (required)
     kafka_key: ${! @kafka_key }
     max_in_flight: 1
@@ -44,7 +44,7 @@ Advanced::
 # All config fields, showing default values
 output:
   label: ""
-  kafka_migrator_offsets:
+  redpanda_migrator_offsets:
     seed_brokers: [] # No default (required)
     kafka_key: ${! @kafka_key }
     client_id: benthos
@@ -549,4 +549,4 @@ The maximum period to wait before retry attempts are abandoned. If set to `0`, t
 
 *Default*: `"30s"`
 
-// end::single-source[]
+// end::single-source[]