FME Python Scripting & Automation
FME Python Scripting & Automation
A
complete guide to embedding Python logic inside FME workspaces using
PythonCaller and PythonCreator transformers — with real-world examples.
|
FME (Feature Manipulation
Engine) has built-in Python support through two special transformers —
PythonCaller and PythonCreator allowing you to embed custom Python logic
directly inside your FME workspaces without leaving the FME environment. |
In this blog, we will explore how
to use Python scripting inside FME, when to use it, real-world examples, and
best practices to keep your workspaces clean and maintainable.
1. What is FME Python Scripting?
FME natively supports hundreds of
transformers for data transformation, but sometimes your logic is too complex
or specific for a standard transformer. This is where Python scripting becomes
invaluable.
FME provides two Python-based transformers:
|
Transformer |
Purpose |
|
PythonCaller |
Processes
features flowing through the workspace — read, modify, and output existing
features |
|
PythonCreator |
Generates
brand-new features from scratch no input features required |
2. When Should You Use Python in FME?
Python scripting inside FME is
best suited for scenarios where built-in transformers fall short:
•
Custom string manipulation
beyond what StringReplacer or AttributeCreator can handle
•
Complex conditional logic
spanning multiple attributes
•
Calling external REST APIs
or web services during the ETL process
•
Reading from or writing to
external databases or files
•
File system operations such
as copying, moving, or renaming files mid-workflow
•
Date/time calculations not
covered by built-in transformers
•
Generating dynamic content
or synthetic test data
3. PythonCaller Process Existing Features
The PythonCaller transformer processes features as they flow through your FME workspace. It is the most commonly used Python transformer.
3.1 Basic Structure
Every PythonCaller script must
define a class with the following three methods:
|
import fme |
|
import fmeobjects |
|
|
|
class FeatureProcessor: |
|
|
|
def __init__(self): |
|
# Called once when the transformer
initialises |
|
pass |
|
|
|
def input(self, feature): |
|
# Called once per feature flowing
through |
|
# Read, modify, and output the
feature here |
|
self.pyoutput(feature) |
|
|
|
def close(self): |
|
# Called once when all features have
been processed |
|
pass |
3.2 Reading and Setting Attributes
The most common operation reading an attribute value, transforming it, and writing it back to the
feature:
|
import fme |
|
import fmeobjects |
|
|
|
class FeatureProcessor: |
|
|
|
def input(self, feature): |
|
# Read an existing attribute |
|
name = feature.getAttribute('NAME') |
|
|
|
# Guard against None (missing
attributes return None) |
|
if name: |
|
feature.setAttribute('NAME_UPPER', name.upper()) |
|
feature.setAttribute('NAME_LENGTH', len(name)) |
|
else: |
|
feature.setAttribute('NAME_UPPER', '') |
|
feature.setAttribute('NAME_LENGTH', 0) |
|
|
|
self.pyoutput(feature) |
|
|
|
def close(self): |
|
pass |
3.3 Calling an External REST API
You can call any external web API
directly from inside an FME workspace using Python's requests library:
|
import fme |
|
import fmeobjects |
|
import requests |
|
|
|
class FeatureProcessor: |
|
|
|
def input(self, feature): |
|
address = feature.getAttribute('ADDRESS') |
|
|
|
try: |
|
response = requests.get( |
|
'https://geocode.example.com/api', |
|
params={'q': address}, |
|
timeout=10 |
|
) |
|
data = response.json() |
|
|
|
feature.setAttribute('LATITUDE',
data.get('lat', '')) |
|
feature.setAttribute('LONGITUDE',
data.get('lon', '')) |
|
feature.setAttribute('API_STATUS', 'SUCCESS') |
|
|
|
except Exception as e: |
|
fme.logMessageString(f'API call
failed: {e}', fme.SEV_WARNING) |
|
feature.setAttribute('API_STATUS', 'FAILED') |
|
|
|
self.pyoutput(feature) |
|
|
|
def close(self): |
|
pass |
3.4 File System Operations
Perform file copy, move, or rename
operations during the FME workflow:
|
import fme |
|
import fmeobjects |
|
import os |
|
import shutil |
|
|
|
class FeatureProcessor: |
|
|
|
def input(self, feature): |
|
src
= feature.getAttribute('SOURCE_PATH') |
|
dest =
feature.getAttribute('DEST_PATH') |
|
|
|
if os.path.exists(src): |
|
shutil.copy(src, dest) |
|
feature.setAttribute('COPY_STATUS', 'SUCCESS') |
|
fme.logMessageString(f'Copied:
{src} -> {dest}') |
|
else: |
|
feature.setAttribute('COPY_STATUS', 'FILE NOT FOUND') |
|
fme.logMessageString(f'Missing:
{src}', fme.SEV_WARNING) |
|
|
|
self.pyoutput(feature) |
|
|
|
def close(self): |
|
pass |
4. PythonCreator Generate New Features
The PythonCreator transformer creates brand-new FME features from scratch. Unlike PythonCaller, it does not require any input features — it is driven entirely by your Python code.
4.1 Basic Structure
|
import fme |
|
import fmeobjects |
|
|
|
class Creator: |
|
|
|
def __init__(self): |
|
# Create and output features in
__init__ |
|
for i in range(1, 4): |
|
feature = fmeobjects.FMEFeature() |
|
feature.setAttribute('ROW_ID',
i) |
|
feature.setAttribute('SOURCE',
'Python') |
|
feature.setAttribute('CREATED',
'2026-04-06') |
|
self.pyoutput(feature) |
|
|
|
def input(self, feature): |
|
# Not used in PythonCreator — leave
empty |
|
pass |
|
|
|
def close(self): |
|
pass |
4.2 Generating Features from a Database
Use PythonCreator to pull data
from an external database and create FME features dynamically:
|
import fme |
|
import fmeobjects |
|
import pyodbc |
|
|
|
class Creator: |
|
|
|
def __init__(self): |
|
conn_str = ( |
|
'DRIVER={ODBC Driver 17 for SQL
Server};' |
|
'SERVER=MYSERVER;DATABASE=MYDB;Trusted_Connection=yes' |
|
) |
|
|
|
try: |
|
conn = pyodbc.connect(conn_str) |
|
cursor = conn.cursor() |
|
cursor.execute('SELECT ID, NAME,
STATUS FROM dbo.ASSETS') |
|
|
|
for row in cursor.fetchall(): |
|
feature =
fmeobjects.FMEFeature() |
|
feature.setAttribute('ASSET_ID',
row.ID) |
|
feature.setAttribute('ASSET_NAME',
row.NAME) |
|
feature.setAttribute('ASSET_STATUS', row.STATUS) |
|
self.pyoutput(feature) |
|
|
|
conn.close() |
|
|
|
except Exception as e: |
|
fme.logMessageString(f'DB error:
{e}', fme.SEV_ERROR) |
|
|
|
def input(self, feature): |
|
pass |
|
|
|
def close(self): |
|
pass |
5. Key FME Python Classes & Methods
Here is a quick reference of the
most commonly used FME Python classes and methods:
|
Class /
Method |
Purpose |
|
fmeobjects.FMEFeature() |
Create a
brand-new FME feature object |
|
feature.getAttribute('NAME') |
Read the
value of an attribute (returns None if missing) |
|
feature.setAttribute('NAME',
value) |
Write or
update an attribute value on a feature |
|
self.pyoutput(feature) |
Send a
feature to the output port |
|
fme.logMessageString('msg') |
Write an INFO
message to the FME log |
|
fme.logMessageString('msg',
fme.SEV_WARNING) |
Write a
WARNING message to the FME log |
|
fme.logMessageString('msg',
fme.SEV_ERROR) |
Write an
ERROR message to the FME log |
|
feature.getGeometry() |
Retrieve the
geometry object attached to a feature |
|
feature.setGeometry(geom) |
Assign a
geometry object to a feature |
6. Installing Third-Party Python Packages
FME ships with its own bundled
Python interpreter. Third-party packages must be installed into FME's Python
environment, not your system Python.
Step 1 Find FME's Python Path
|
# Inside an FME
PythonCaller or PythonCreator: |
|
import sys |
|
fme.logMessageString(sys.executable) # logs FME's Python path |
Step 2 — Install Package Using FME's pip
|
# Open Command Prompt as
Administrator |
|
# Navigate to FME's
Python Scripts folder |
|
|
|
cd "C:\Program
Files\FME\python" |
|
|
|
# Install into FME's
environment |
|
python.exe -m pip
install requests |
|
python.exe -m pip
install pyodbc |
|
Important: Installing
packages into your system Python will NOT make them available inside FME
workspaces. Always install into the FME Python interpreter. |
7. Best Practices
•
Always handle None values
from getAttribute() FME returns None for missing or null attributes, which
will cause AttributeError if not checked
•
Use fme.logMessageString()
for debugging print() output does not appear in the FME log window
•
Keep Python logic focused
and lightweight heavy processing belongs in standalone Python scripts called
via the PythonCaller or ShellCreator
•
Wrap external calls in
try/except blocks network timeouts, missing files, or DB errors should be
caught and logged gracefully
•
Use the close() method for
cleanup tasks such as closing database connections or file handles
•
Test your Python class in
isolation before embedding it in an FME workspace — use a simple test harness
outside FME first
8. PythonCaller vs PythonCreator Quick Comparison
|
Feature |
PythonCaller |
PythonCreator |
|
Input
features required? |
Yes |
No |
|
Triggered by |
Each feature
flowing through |
__init__
method on startup |
|
Best used for |
Modifying
existing features |
Creating new
features from scratch |
|
Common use
cases |
Attribute
transforms, API calls, file ops |
DB reads,
synthetic data generation |
|
Output method |
self.pyoutput(feature) |
self.pyoutput(feature) |
9. Summary
|
Python scripting in FME
unlocks capabilities far beyond what built-in transformers can provide.
Whether you are calling external APIs, working with file systems, querying
databases, or building complex business logic — PythonCaller and
PythonCreator give you the flexibility to handle it all, directly inside your
FME workflow. |
In the next blog, we will cover
Blog #12 — FME Batch Workspace Runner Guide, where we will walk through
automating the sequential execution of multiple FME workspaces using Python and
Windows Task Scheduler.
Tags:
FME |
Python | PythonCaller
| PythonCreator | GIS
Automation | ETL
| ArcGIS
Comments
Post a Comment