getArrayLength() is used to return an integer number representing the total number of rows that compose the DISPLAY ARRAY or INPUT ARRAY lists. The method needs the screen array name as an argument:
DIALOG
INPUT ARRAY custlist FROM sa_custlist.*
BEFORE ROW
MESSAGE "Row count: " || DIALOG.getArrayLength("sa_custlist")
...
END INPUT
There is some specifics about using variables with ui.Dialog methods.
You can learn about them here.