svn: inconsistent line ending style

Today I was adding grails tutorials into SVN repository. Yes I know it should be in the repository long time ago :). And something that should be simple operation finished with the svn:inconsistent line ending style. For those who didn't still hit this problem, SVN finishes with this error if you have different line ending styles in the same file. And it will refuse to add such files into repository till it is not fixed. As there was more than one file with such problem (few hundreds of them) manual intervention was not an option. But to my surprise (after googling) I was not able to find how to fix it automatically for all the files. So I decided to write a groovy script that will fix it for me.

And without too much waiting groovy script is here:

if (!args) {
   println "Usage: <path_to_directory>"
   println "And: <existance of extensions.txt comma separated values file with file extensions to convert>"
   return
}

Convert c = new Convert()
c.convert(args[0])


class Convert {
   Set extensions = new HashSet()
  
   public void convert(String path) {
       extensions()
      
       println "extensions to convert: ${extensions}"

       File f = new File(path)
       convertDir(f)
   }
      
   private void convertDir(File f) {
       def sum = 0
       def filesFound = 0
       def filesVisited = 0
       f.eachFileRecurse{File file ->
           filesVisited = filesVisited + 1
           if (shouldConvert(file)) {
               filesFound = filesFound + 1
               if (filesFound % 100 == 0) {
                   println "Files checked: ${filesFound}"
               }
               if (replaceLines(file)) {
                   sum = sum + 1
                   if (sum % 10 == 0)
                   println "Replaced eol in files: " + sum
               }
           }
       }
      
       println "Files converted: ${sum}"
       println "Files checked: ${filesFound}"
       println "Files visited: ${filesVisited}"
   }
  
   private boolean shouldConvert(File f) {
       return extensions.contains(extension(f))
   }
  
   private String extension(File f) {
       int idx = f.getName().lastIndexOf('.')
       if (idx != -1) {
           String result = f.getName().substring(idx + 1)
           return result
       } else {
           return null
       }
   }
  
   def replaceLines = {File f ->
       String text = f.text
      
       if (text.contains('\r\n') || text.contains('\r')) {
           text = text.replaceAll('\r\n', '\n')
           text = text.replaceAll('\r', '\n')
           f.write(text)
           return true
       }
       return false
   }
  
   private void extensions() {
       File f = new File("extensions.txt")
       String content = f.text
       String[] str = content.split(",")
       Set extensions = new HashSet(Arrays.asList(str))
       extensions.each{
           it = it.replaceAll('\r\n','')
           this.extensions.add(it)
       }
   }
}

To be able to use this script you have to install groovy. Then in the same directory where you have your file you need to create extensions.txt file that contains list of file extensions that should be checked. Extensions should be comma separated without spaces in between.

Then run the script with groovy convert.groovy <path_to_directory>.

Now what is visible in this simple script is how groovy extensions to the java.io.File help us to work with files and directories. Actually if you go through code you will see that following methods have been used:

• f.eachFileRecurse - will recursively traverse of files in the directory structure


• f.text - will return you content of the whole file as String


• f.write - will write string as a context to the file

Well you can use this script if you have the same problem but you know that script is provided as is and in the case of damage I will not feel responsible any way.

Grails, Groovy, XML

Last few days, while working on grails tutorials, I needed to translate some Grails domain objects to XML. I thought it should not be so hard as there is lot of articles about Groovy and XML. But what I found out is that most of the articles present only the simple usage of Groovy XmlSlurper. One of the examples would be:

def xml = new groovy.xml.MarkupBuilder()
xml.person(id:99){
  firstname("John" )
  lastname("Smith" )
}

And what I need is to traverse recursive structure of domain object. I know it is not so hard but anyway I decided to describe it here. At the end, as you will see it is very simple. But, by the way I will also explain how Grails supports XML transformations.

Let start with Grails support. If you don't request special structure of XML but it is enough to directly transform domain object to XML then Grails is excellent solution for you.

Domain object to translate is:

class LearningArea {
  String title
  String description
  Date dateCreated
  Date lastUpdated
  List tutorials
  List subAreas

  static hasMany = [tutorials: TutorialLink, subAreas: LearningArea]

  static constraints = {
      title(blank:false, minSize:3, maxSize:500)
              description(blank:false, maxSize:3000)
  }

  public String toString() {
      return title
  }
}

In Grails you can translate object to XML as in following example:

LearningArea learningArea = LearningArea.get(1)
String xml = render learningArea as XML

But interesting thing to notice is that render can be imported from two packages:

grails.converters.XML and grails.converters.deep.XML

And of course there is difference which package you use.

If you use deep package complete domain objects tree (with all sub areas's, sub areas and so on...) will be generated with all the properties. So in the case of LearningArea complete tree of all subAreas and all tutorials will be translated to XML including all properties.

If you use just converters.XML package, top level object is fully translated but referenced objects are translated only to the level of ids. This means that XML generated will be much smaller but to obtain additional information you need to retrieve additional data later.

Unfortunately for me, I couldn't use this Grails XML magic because I didn't want tutorials in the XML but I wanted full tree of subAreas only with titles and without description properties. So I had to use Groovy XmlSlurper. Important thing to notice in the above example of XmlSlurper is that when using XML builder, you can also provide closures. So all I needed was to recursively traverse the tree of the subAreas. And the code to do that is very simple:

    def generateXML= {
      def writer = new StringWriter()
      MarkupBuilder xml = new MarkupBuilder(writer)

      LearningArea lr = LearningArea.findByTitle("Groovy")

      toXml(lr, xml)

      String str = writer.toString()
  }

  private void toXml(LearningArea la, def xml) {
      xml.learningArea(name: la.title, id: la.id) {
          la.subAreas.each {
              toXml(it, xml)
          }
      }
  }

As we can see all the logic is in the toXml method. This method accepts instance of LearningArea and MarkupBuilder, writes element learningArea to XML with attributes name and id, and the as subelements recursively writes all sub learning areas. This way the complete XML tree is generated in few lines of code and I got example that is little bit more complex than most of the examples on the web.

And if you didn't visited grails tutorials yes, you can do it now :)