Split Join

lua-users home

"split" [1] and "join" [2] are two common string operators that are essentially inverse operations of each other. Split separates a string containing a delimiter into the list of substrings between that delimiter. Join combines a list of strings into a new string by inserting a delimiter between each string.

There are various ways to design and implement these functions in Lua, as described below.

Joining list of strings

With Lua 5.x you can use table.concat[3] for joining: table.concat(tbl, delimiter_str).

table.concat({"a", "b", "c"}, ",") --> "a,b,c"

Other interfaces are possible, largely dependent on the choice of split interface since join is often intended to be the inverse operation of split.

Splitting Strings

First of all, although Lua does not have a split function in its standard library, it does have string.gmatch[4], which can be used instead of a split function in many cases. Unlike a split function, string.gmatch takes a pattern to match the non-delimiter text, instead of the delimiters themselves:

local example = "an example string"
for i in string.gmatch(example, "%S+") do

-- output:
-- an
-- example
-- string

A split[1] function separates a string into a list of substrings, breaking the original string on occurrences of some separator (character, character set, or pattern). There are various ways to design a string split function. A summary of the design decisions is listed below.

Should split return a table array, a list, or an iterator?

split("a,b,c", ",") --> {"a", "b", "c"}
split("a,b,c", ",") --> "a","b","c" (not scalable: Lua has a limit of a few thousand return values)
for x in split("a,b,c", ",") do ..... end

Should the separator be a string, Lua pattern, LPeg pattern, or regular expression?

split("a  +b c", " +") --> {"a ", "b c"}
split("a  +b c", " +") --> {"a", "+b", "c"}
split("a  +b c", some_other_object) --> .....

How should empty separators be handled?

split("abc", "") --> {"a", "b", "c"} 
split("abc", "") --> {"", "a", "b", "c", ""}
split("abc", "") --> error
split("abc", "%d*") --> what about patterns that can evaluate to empty strings?

Note: split(s,"") is a convenient idiom for splitting a string into characters. In Lua, we can alternately do for c in s:gmatch"." do ..... end.

How should empty values be handled?

split(",,a,b,c,", ",") --> {"a", "b", "c"}
split(",,a,b,c,", ",") --> {"", "", "a", "b", "c", ""}
split(",", ",") --> {} or {""} or {"", ""} ?
split("", ",") --> {} or {""} ?

Note: Although splitting and joining are roughly inverses, the operations are not necessarily uniquely determined, particularly when there are empty strings. join({"",""}, ""), join({""}, "") and join({}, "") all result in the same string "". Therefore, the choice of what the inverse operation split("", "") should return is not immediately clear.

Note: completely ignoring empty values can be undesirable, such as for rows in a CSV file where column positions matter. A CSV file with empty rows "" is unclear: is this an column containing an empty value or are there zero columns? A zero column CSV file is unlikely though perhaps not impossible.

Should there be an argument to limit the number of splits?

split("a,b,c", ",", 2) --> {"a", "b,c"}

Should the separator be returned? This is more useful when the separator is a pattern, in which case the separator can vary:

split("a  b c", " +") --> {"a", "  ", "b", " ", "c"}

Note: Note also that string.gmatch [5] is in a way a dual of split, returning the substrings that match a pattern and discarding strings between them rather than the other way around. A function that returns both is sometimes called partition [6].

Method: Split by pattern using string.gsub/string.match

Break a string up at occurrences of a single character. If the number of fields is known:

str:match( ("([^"..sep.."]*)"..sep):rep(nsep) )

If the number of fields is not known

fields = {str:match((str:gsub("[^"..sep.."]*"..sep, "([^"..sep.."]*)"..sep)))}

Some might call the above a hack :) sep will need to be escaped if it is a pattern metacharacter, and you'd probably be better off precomputing and/or memorizing the patterns. And it leaves off values after the last separator. E.g. "a,b,c" returns "a" and "b" but not "c"

Method: Using only string.gsub

fields = {}
str:gsub("([^"..sep.."]*)"..sep, function(c)
   table.insert(fields, c)

Does not work as expected:

str, sep = "1:2:3", ":"
fields = {}
str:gsub("([^"..sep.."]*)"..sep, function(c)
   table.insert(fields, c)
for i,v in ipairs(fields) do

-- output:
-- 1        1
-- 2        2


function string:split(sep)
   local sep, fields = sep or ":", {}
   local pattern = string.format("([^%s]+)", sep)
   self:gsub(pattern, function(c) fields[#fields+1] = c end)
   return fields

Example: split a string into words, or return nil

function justWords(str)
   local t = {}
   local function helper(word)
      table.insert(t, word)
      return ""
   if not str:gsub("%w+", helper):find"%S" then
      return t

Method: Split a string with a pattern, Take One

This splits a string using the pattern sep. It calls func for each segment. When func is called, the first argument is the segment and the remaining arguments are the captures from sep, if any. On the last segment, func will be called with just one argument. (This could be used as a flag, or you could use two different functions). sep must not match the empty string. Enhancements are left as an exercise :)

func((str:gsub("(.-)("..sep..")", func)))

Example: Split a string into lines separated by either DOS or Unix line endings, creating a table out of the results.

function lines(str)
   local t = {}
   local function helper(line)
      table.insert(t, line)
      return ""
   helper((str:gsub("(.-)\r?\n", helper)))
   return t

Function: Split a string with a pattern, Take Two

The problem with using gsub as above is that it can't handle the case when the separator pattern doesn't appear at the end of the string. In that case the final "(.-)" never gets to capture the end of the string, because the overall pattern fails to match. To handle that case you have to do something a little more complicated. The split function below behaves more or less like split in perl or python. In particular, single matches at the beginning and end of the string do not create new elements. Multiple matches in a row create empty string elements.

-- Compatibility: Lua-5.1
function split(str, pat)
   local t = {}  -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pat
   local last_end = 1
   local s, e, cap = str:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
      last_end = e+1
      s, e, cap = str:find(fpat, last_end)
   if last_end <= #str then
      cap = str:sub(last_end)
      table.insert(t, cap)
   return t

Example: Split a file path string into components.

function split_path(str)
   return split(str,'[\\/]+')

parts = split_path("/usr/local/bin")
--> {'usr','local','bin'}

Test Cases:

--> {'foo','bar','baz','test'}
--> {'foo','bar','baz','test'}
--> {'foo','bar','baz','test'}
--> {'foo','bar','','baz','test','',''}
--> {'foo','bar','baz','test'}
--> {'foo'}
--> {}
split('foo','')  -- opps! infinite loop!

Function: Split a string with a pattern, Take Three

After a discussion on this topic in the mailing list, I made my own function... I took, unknowingly, a way similar to the function above, except I use gfind to iterate, and I see the single matches at beginning and end of string as empty fields. As above, multiple successive delimiters create empty string elements.

-- Compatibility: Lua-5.0
function Split(str, delim, maxNb)
   -- Eliminate bad cases...
   if string.find(str, delim) == nil then
      return { str }
   if maxNb == nil or maxNb < 1 then
      maxNb = 0    -- No limit
   local result = {}
   local pat = "(.-)" .. delim .. "()"
   local nb = 0
   local lastPos
   for part, pos in string.gfind(str, pat) do
      nb = nb + 1
      result[nb] = part
      lastPos = pos
      if nb == maxNb then
   -- Handle the last field
   if nb ~= maxNb then
      result[nb + 1] = string.sub(str, lastPos)
   return result

Test Cases:

ShowSplit("abc", '')
--> { [1] = "", [2] = "", [3] = "", [4] = "", [5] = "" }
-- No infite loop... but garbage in, garbage out...
ShowSplit("", ',')
--> { [1] = "" }
ShowSplit("abc", ',')
--> { [1] = "abc" }
ShowSplit("a,b,c", ',')
--> { [1] = "a", [2] = "b", [3] = "c" }
ShowSplit("a,b,c,", ',')
--> { [1] = "a", [2] = "b", [3] = "c", [4] = "" }
ShowSplit(",a,b,c,", ',')
--> { [1] = "", [2] = "a", [3] = "b", [4] = "c", [5] = "" }
ShowSplit("x,,,y", ',')
--> { [1] = "x", [2] = "", [3] = "", [4] = "y" }
ShowSplit(",,,", ',')
--> { [1] = "", [2] = "", [3] = "", [4] = "" }
ShowSplit("x!yy!zzz!@", '!', 4)
--> { [1] = "x", [2] = "yy", [3] = "zzz", [4] = "@" }
ShowSplit("x!yy!zzz!@", '!', 3)
--> { [1] = "x", [2] = "yy", [3] = "zzz" }
ShowSplit("x!yy!zzz!@", '!', 1)
--> { [1] = "x" }

ShowSplit("a:b:i:p:u:random:garbage", ":", 5)
--> { [1] = "a", [2] = "b", [3] = "i", [4] = "p", [5] = "u" }
ShowSplit("hr , br ;  p ,span, div", '%s*[;,]%s*')
--> { [1] = "hr", [2] = "br", [3] = "p", [4] = "span", [5] = "div" }


Function: Perl-like split/join

Many people miss Perl-like split/join functions in Lua. Here are mine:

-- Concat the contents of the parameter list,
-- separated by the string delimiter (just like in perl)
-- example: strjoin(", ", {"Anna", "Bob", "Charlie", "Dolores"})
function strjoin(delimiter, list)
   local len = getn(list)
   if len == 0 then
      return "" 
   local string = list[1]
   for i = 2, len do 
      string = string .. delimiter .. list[i] 
   return string

-- Split text into a list consisting of the strings in text,
-- separated by strings matching delimiter (which may be a pattern). 
-- example: strsplit(",%s*", "Anna, Bob, Charlie,Dolores")
function strsplit(delimiter, text)
   local list = {}
   local pos = 1
   if strfind("", delimiter, 1) then -- this would result in endless loops
      error("delimiter matches empty string!")
   while 1 do
      local first, last = strfind(text, delimiter, pos)
      if first then -- found?
         tinsert(list, strsub(text, pos, first-1))
         pos = last+1
         tinsert(list, strsub(text, pos))
   return list


Function: Perl-like split/join, alternative

Here's my own split function, for comparison. It's largely the same as the above; not quite as DRY but (IMO) slightly cleaner. It doesn't use gfind (as suggested below) because I wanted to be able to specify a pattern for the split string, not a pattern for the data sections. If speed is paramount, it might be made faster by caching string.find as a local 'strfind' variable, as the above does.

--Written for 5.0; could be made slightly cleaner with 5.1
--Splits a string based on a separator string or pattern;
--returns an array of pieces of the string.
--(May optionally supply a table as the third parameter which will be filled 
with the results.)
function string:split( inSplitPattern, outResults )
   if not outResults then
      outResults = { }
   local theStart = 1
   local theSplitStart, theSplitEnd = string.find( self, inSplitPattern, 
theStart )
   while theSplitStart do
      table.insert( outResults, string.sub( self, theStart, theSplitStart-1 ) )
      theStart = theSplitEnd + 1
      theSplitStart, theSplitEnd = string.find( self, inSplitPattern, theStart )
   table.insert( outResults, string.sub( self, theStart ) )
   return outResults


Function: PHP-like explode

Explode string into table with seperator (moved from TableUtils):

-- explode(seperator, string)
function explode(d,p)
   local t, ll
   if(#p == 1) then
      return {p}
   while true do
      l = string.find(p, d, ll, true) -- find the next d in the string
      if l ~= nil then -- if "not not" found then..
         table.insert(t, string.sub(p,ll,l-1)) -- Save it in our array.
         ll = l + 1 -- save just after where we found it for searching next time.
         table.insert(t, string.sub(p,ll)) -- Save what's left in our array.
         break -- Break at end, as it should be, according to the lua manual.
   return t

Here's my version of PHP style explode, supporting limit

function explode(sep, str, limit)
   if not sep or sep == "" then
      return false
   if not str then
      return false
   limit = limit or mhuge
   if limit == 0 or limit == 1 then
      return {str}, 1

   local r = {}
   local n, init = 0, 1

   while true do
      local s,e = strfind(str, sep, init, true)
      if not s then
      r[#r+1] = strsub(str, init, s - 1)
      init = e + 1
      n = n + 1
      if n == limit - 1 then

   if init <= strlen(str) then
      r[#r+1] = strsub(str, init)
      r[#r+1] = ""
   n = n + 1

   if limit < 0 then
      for i=n, n + limit + 1, -1 do r[i] = nil end
      n = n + limit

   return r, n
(Lance Li)

Function: Using metatable and __index

This function uses a metatable's __index function to populate the table of split parts. This function does not try to (correctly) invert the pattern, and so really doesn't work as most string split functions do.

--[[ written for Lua 5.1
split a string by a pattern, take care to create the "inverse" pattern 
yourself. default pattern splits by white space.
string.split = function(str, pattern)
   pattern = pattern or "[^%s]+"
   if pattern:len() == 0 then
      pattern = "[^%s]+"
   local parts = {__index = table.insert}
   setmetatable(parts, parts)
   str:gsub(pattern, parts)
   setmetatable(parts, nil)
   parts.__index = nil
   return parts
-- example 1
str = "no separators in this string"
parts = str:split( "[^,]+" )
print( # parts )
table.foreach(parts, print)
--[[ output:
1	no separators in this string

-- example 2
str = "   split, comma, separated  , , string   "
parts = str:split( "[^,%s]+" )
print( # parts )
table.foreach(parts, print)
--[[ output:
1	split
2	comma
3	separated
4	string

Function: true Python semantics for split

This is the Python behavior:

Python 2.5.1 (r251:54863, Jun 15 2008, 18:24:51) 
[GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2
>>> 'x!yy!zzz!@'.split('!')
['x', 'yy', 'zzz', '@']
>>> 'x!yy!zzz!@'.split('!', 3)
['x', 'yy', 'zzz', '@']
>>> 'x!yy!zzz!@'.split('!', 2)
['x', 'yy', 'zzz!@']
>>> 'x!yy!zzz!@'.split('!', 1)
['x', 'yy!zzz!@']

And IMHO this Lua function implements this semantics:

function string:split(sSeparator, nMax, bRegexp)
   assert(sSeparator ~= '')
   assert(nMax == nil or nMax >= 1)

   local aRecord = {}

   if self:len() > 0 then
      local bPlain = not bRegexp
      nMax = nMax or -1

      local nField, nStart = 1, 1
      local nFirst,nLast = self:find(sSeparator, nStart, bPlain)
      while nFirst and nMax ~= 0 do
         aRecord[nField] = self:sub(nStart, nFirst-1)
         nField = nField+1
         nStart = nLast+1
         nFirst,nLast = self:find(sSeparator, nStart, bPlain)
         nMax = nMax-1
      aRecord[nField] = self:sub(nStart)

   return aRecord

Observe the possibility to use simple strings or regular expressions as delimiters.

Test Cases:

Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> for k,v in next, string.split('x!yy!zzz!@', '!') do print(v) end
> for k,v in next, string.split('x!yy!zzz!@', '!', 3) do print(v) end
> for k,v in next, string.split('x!yy!zzz!@', '!', 2) do print(v) end
> for k,v in next, string.split('x!yy!zzz!@', '!', 1) do print(v) end


If we define split simply as "return all 0-n character occurences followed by a separator, plus what's left of the string", which I think results in the most intuitive splitting logic, then we get a simple implementation using just gmatch which covers all cases and still allows the separator to be a pattern:

function gsplit(s,sep)
   return coroutine.wrap(function()
      if s == '' or sep == '' then
      local lasti = 1
      for v,i in s:gmatch('(.-)'..sep..'()') do
         lasti = i

--same idea without coroutines

function gsplit2(s,sep)
   local lasti, done, g = 1, false, s:gmatch('(.-)'..sep..'()')
   return function()
      if done then
      local v,i = g()
      if s == '' or sep == '' then
         done = true
         return s
      if v == nil then
         done = true
         return s:sub(lasti)
      lasti = i
      return v

The gsplit() above returns an iterator, so other API variants can be easily derived from it:

function iunpack(i,s,v1)
   local function pass(...)
      local v1 = i(s,v1)
      if v1 == nil then
         return ...
      return v1, pass(...)
   return pass()

function split(s,sep)
   return iunpack(gsplit(s,sep))

function accumulate(t,i,s,v)
   for v in i,s,v do
      t[#t+1] = v
   return t

function tsplit(s,sep)
   return accumulate({}, gsplit(s,sep))

Note that the above implementation does not allow captures in the separator. To allow for that, another closure must be created to pass along the additional captured strings (see VarargTheSecondClassCitizen). The semantics also get muddy (I suppose one use case would be wanting to know what the actual separator was for each string, eg. for a separator pattern like [%.,;]).

function gsplit(s,sep)
   local i, done, g = 1, false, s:gmatch('(.-)'..sep..'()')
   local function pass(...)
      if ... == nil then
         done = true
         return s:sub(i)
      i = select(select('#',...),...)
      return ...
   return function()
      if done then
      if s == '' or sep == '' then
         done = true
         return s
      return pass(g())

The problem with the above implementation is that however easy to read, the (.-) pattern in Lua has awful performance, hence the following implementation based on only string.find (allows for captures in the separator and adds a third argument "plain", similar to string.find):

function string.gsplit(s, sep, plain)
   local start = 1
   local done = false
   local function pass(i, j, ...)
      if i then
         local seg = s:sub(start, i - 1)
         start = j + 1
         return seg, ...
         done = true
         return s:sub(start)
   return function()
      if done then
      if sep == '' then
         done = true
         return s
      return pass(s:find(sep, start, plain))

Unit testing:

local function test(s,sep,expect)
   local t={} for c in s:gsplit(sep) do table.insert(t,c) end
   assert(#t == #expect)
   for i=1,#t do assert(t[i] == expect[i]) end
   test(t, expect)
test('', ',', {''})
test(',', ',', {'',''})
test('a', ',', {'a'})
test('a,b', ',', {'a','b'})
test('a,b,', ',', {'a','b',''})
test(',a,b', ',', {'','a','b'})
test(',a,b,', ',', {'','a','b',''})
test(',a,,b,', ',', {'','a','','b',''})
test('a,,b', ',', {'a','','b'})
test('asd  ,   fgh  ,;  qwe, rty.   ,jkl', '%s*[,.;]%s*', {'asd','fgh','','qwe','rty','','jkl'})
test('Spam eggs spam spam and ham', 'spam', {'Spam eggs ',' ',' and ham'})


-- single char string splitter, sep *must* be a single char pattern
-- *probably* escaped with % if it has any special pattern meaning, eg "%." not "."
-- so good for splitting paths on "/" or "%." which is a common need

local function csplit(str,sep)
   local ret={}
   local n=1
   for w in str:gmatch("([^"..sep.."]*)") do
      ret[n] = ret[n] or w -- only set once (so the blank after a string is ignored)
      if w=="" then
         n = n + 1
      end -- step forwards on a blank but not a string
   return ret

-- the following is true of any string, csplit will do the reverse of a concat

local str=""
print(str , assert( table.concat( csplit(str,"/") , "/" ) == str ) )

local str="only"
print(str , assert( table.concat( csplit(str,"/") , "/" ) == str ) )

local str="/test//ok/"
print(str , assert( table.concat( csplit(str,"/") , "/" ) == str ) )

local str=".test..ok."
print(str , assert( table.concat( csplit(str,"%.") , "." ) == str ) )

Semantic changes in Lua 5.3.3

Up to Lua 5.3.2, splitting was tricky in most situations, because string.gmatch and string.gsub introduce spurious extra empty fields (as in Perl). From Lua 5.3.3 onwards, they no longer do that, they now behave as in Python. Therefore, the following minimalistic splitting function now is a true inverse of table.concat; previously it was not.

--- splits 'str' into pieces matching 'pat', returns them as an array
local function split(str,pat)
  local tbl={}
  str:gsub(pat,function(x) tbl[#tbl+1]=x end)
  return tbl

local str = "a,,b"     -- comma-separated list
local pat = "[^,]*"    -- everything except commas
assert (table.concat(split(str,pat),",") == str)


Comparison to Other Languages

User Comments

I mean no disrespect, of course, but.. does anyone actually have a working split function without glitches like infinite loops, wrong matches, or error cases? Are all those "takes" of any help here? -- CosminApreutesei

Try Rici Lake's split function: LuaList:2006-12/msg00414.html -- Jörg Richter

That version fails again when the pattern is an empty string. Specifications for split functions in other languages define how these corner cases should behave (see "Comparison to Other Languages" above). --David Manura

See Also

RecentChanges · preferences
edit · history
Last edited September 18, 2017 8:39 am GMT (diff)