13

Assume somebody familiar with DevOps and mainframes, but new to Jenkins, wants to get started with Jenkins, e.g. to:

After some kind of (minimal) Jenkins installation and configuration got completed, the actual question becomes "Which of the typical Jenkins plugins, if any, would be needed?". From the various Jenkins plugins that appear to make sense in my case, it seems that these are possible candidates (quotes are from the linked pages):

  • IBM zOS Connector.

    ... provides its functions via FTP connection to IBM z/OS LPAR. You can configure your SCLM project on z/OS and then check for the changes via Jenkins.

    Features include:

    • Submission of user JCL job (with optional log collected upon finish).
    • Introduction of SCLM as SCM for your projects allowing to checkout SCLM changes.
    • The ability to build SCLM projects currently can be performed only via 'Submit zOS Job' build action.

  • Compuware Source Code Download for Endevor, PDS, and ISPW Plugin.

    ... allows Jenkins users to download Endevor, PDS, or ISPW members from the mainframe to the PC. Source can then be accessed on the PC, for example, for SonarQube analysis and reporting.

Since the 1st plugin is about SCLM (which is all about PDSes), and the 2nd plugin explicitly states support for PDSes (in its name), I trust both are possible candidates.

So assuming my shortlist is complete (is it?), what are the pros and cons of either one that should help me decide which one is best for my case?

Ps: Something like a "Jenkins Kickstart" package doesn't seem to exist (at least I've not found it yet).

Community
  • 1
Pierre.Vriens
  • 7,205
  • 14
  • 37
  • 84
  • 1
    By searching, a lot of searching, and maybe asking here when there's a pitfall on a specific subject. While building a resource of Q/A is great, that really too broad as is. – Tensibai Mar 03 '17 at 14:48
  • 1
    The question would be easier to answer if you would to outline a problem you want to solve with Jenkins. Trying to solve a specific problem is also a good start to learn a new tool! – Michaël Le Barbier Mar 03 '17 at 14:58
  • @MichaelGrünewald : how good are you in "mainframes" (is my question specific enough now)? – Pierre.Vriens Mar 03 '17 at 16:59
  • @Pierre.Vriens really, start by Jenkins website . There's links and tutorials for the ´kickstart'. For the rest there's too much for one question IMHO and without a clear problem statement and indices you already tried by yourself I am afraid this would be badly received – Tensibai Mar 03 '17 at 17:01
  • 1
    @Pierre.Vriens Seems far better now; thanks for editing. I've voted to reopen. – Aurora0001 Mar 03 '17 at 17:03
  • You re question is more or less, how do I install Jenkins and connect it to a specific SCM. If worded around I" can't find a way to make Jenkins interact with this SCM, I am open to change SCM if it support mainframe objects (your link). Is there a specific plugin I overlooked or a tweak possible to achieve that ?" It would be more straight to the point and better I think – Tensibai Mar 03 '17 at 17:05
  • I overlooked the no actual SCM part, sorry. The idea still stand :) make a first choice and try, it's easier to give good answer to focused question – Tensibai Mar 03 '17 at 17:07
  • @Tensibai : merci for your attempt to help, but I'm confused (sorry). I trust you agree I'm not asking for "links to recommend a book, software or getting started docu" (= off topic on most SE sites). And I did already do some homework about the various plugins which IMO won't help (or will be overkill), which is why I added the "not managed by any SCM tool". So would you agree that if I further edit my question to "Is there a specific plugin I overlooked or a tweak possible to achieve that ?" I'm close to an acceptable question? – Pierre.Vriens Mar 03 '17 at 17:13
  • Well if you think no plugin would really help from your experience and that adding a SCM would be overkill for your workflow because it seems to integrate poorly yes. The part really bothering me is the basic setup of Jenkins, I read it as base install of the tool and that sounds too much for me – Tensibai Mar 03 '17 at 17:17
  • Mainly Jenkins is good at doing scripted jobs needed to build a software and trigger things, and I am a little lost about where you are stuck – Tensibai Mar 03 '17 at 17:19
  • Hang on @Tensibai ... You probably don't have an idea where (a) I'm coming from and (b) where I want to arrive "some day". Right now I'm working on further improving "this" question. And while trying to do so, I'm digesting some of your prior comments also. – Pierre.Vriens Mar 03 '17 at 17:27
  • 1
    After réflexion, just don't try to generalize too much your question , describe the problem you're facing and where you're facing a wall , I think that's the best way. – Tensibai Mar 03 '17 at 17:28
  • Sorry :) still unsure if the question is baked as an example or as a real problem... I'd prefer the later , we all have experience of hard things we had to overcome and it's usually giving better real world questions for the site (still my opinion) – Tensibai Mar 03 '17 at 17:31
  • Sorry, one More comment to say I vote to reopen, Previous comments was more general idea about it. I feel I've probably been harsh, it was not my intent. – Tensibai Mar 03 '17 at 17:39
  • 1
    @Tensibai merci for all the comments (attempts to help) ... Yes "a bit" harsh, but don't worry too much about that (I can handle that). Apart from that: hang on for a new attempt to improve (narrow down) my question. – Pierre.Vriens Mar 03 '17 at 17:45
  • WoW, nice rewrites, but I still have no clue of what you want to with Jenkins, linting ? Unit tests ? Isolated integration test ? Some kind of delivery pipeline ? Drawback of Jenkins is how malleable it is, With enough efforts you can achieve whatever needed. In case you're wondering, we implemented linting for RPG and a delivery pipeline on 10 'os400' partitions by driving Arcad through jt400 here, mainframe (zOs) is slightly different, but I still wonder What Is the goal ? :) Side note:I absolutely understand how hard it is to write a question about something you have solved. – Tensibai Mar 03 '17 at 22:27
  • Comments are not a great way to discuss, would be happy to chat to better express my concerns and discuss the topic . I'm not in position to edit in what I mean, because I don't know your goal and have no experience on mainframes CI so I Would probably forget parts. – Tensibai Mar 03 '17 at 22:31

4 Answers4

2

Jenkins slave runs in unix system services of z/OS and can run any script or function from there. We use the Git plugin to pull the source code onto z/OS then you have different choices of how to work with it. You can use IBM Dependency Based Build which is Groovy ported to z/OS plus some extensions to specifically work with native z/OS functions such as copy to PDS (or PDSE) or copy back to zFS, as well as the ability to run MVS, TSO, ISPF commands and submit JCL.

Also with the new Zowe announcement Zowe.org for a new open source project on z/OS you also have access to a number of REST based APIs work with files and perform other tasks on z/OS such as calling zOSMF.

Rosalind Radcliffe
  • 466
  • 3
  • 5
  • I plan to do this: store all mainframe files in GitHub, and install Jenkins slave in USS in z/OS. During a commit or pull request, the webhook in GitHub will kick-off a Jenkins job which will downloads the file from GitHub and submit to z/OS for compile and link (we have perl scripts to do this now). What do you think? – Jirong Hu Jun 21 '19 at 18:30
2

I set up jenkins to manage mainframe tables. Essentially utilizing groovy script plugin. I wrote code that handles movement of files on the mainframe and call them though the plugin. Essentially leveraging the mainframes API to connect and the vsc's commands to move the files.

Eddie
  • 121
  • 2
1

In general, there are two ways to automate actions from Jenkins:

  1. Install a plugin for Jenkins and use it in steps to automate tasks.
  2. Write a script (shell, python, other) and call it from a step to automate tasks.

There are drawbacks / benefits to both approaches. While #1 - the plugin approach - gives you out of the box functionality and does not require you to script, it may be rigid and limited in its abilities. #2 - the scripting approach - requires you to write scripts, as well as maintain them, but may provide much more flexibiity and freedom in automating things.

The scripting approach generally uses a lot of CLIs - aws cli, docker cli, etc.

If #2 - the scripting approach - is viable and you're comfortable with writing shell scripts or other scripts like python to interact with the mainframe, then Brightside can help.

Here's an example of using Brightside (free CLI) to interact with PDS on z/OS from a shell script:

    #!/bin/sh
    echo 'bright files create classic "USER.PUBLIC.SAMPLE.PDS"'
    bright files create classic "USER.PUBLIC.SAMPLE.PDS"
    echo ''

    echo 'bright files download all-members "USER.LIB.SAMP" -d MyPDS'
    bright files download all-members "USER.LIB.SAMP" -d MyPDS
    echo ''

    echo 'bright files upload dir-to-pds MyPDS "USER.PUBLIC.SAMPLE.PDS"'
    bright files upload dir-to-pds MyPDS "USER.PUBLIC.SAMPLE.PDS"
    echo ''

    echo 'bright files list all-members "USER.PUBLIC.SAMPLE.PDS"'
    bright files list all-members "USER.PUBLIC.SAMPLE.PDS"
    echo ''

    echo 'bright files delete data-set "USER.PUBLIC.SAMPLE.PDS(ALLOCLIB)" -f'
    bright files delete data-set "USER.PUBLIC.SAMPLE.PDS(ALLOCLIB)" -f
    echo ''

    echo 'bright files list all-members "USER.PUBLIC.SAMPLE.PDS"'
    bright files list all-members "USER.PUBLIC.SAMPLE.PDS"
    echo ''

    echo 'bright files delete data-set "USER.PUBLIC.SAMPLE.PDS" -f'
    bright files delete data-set "USER.PUBLIC.SAMPLE.PDS" -f
    echo ''

Here's the output from that script:

enter image description here enter image description here

Easy enough to call these scripts from Jenkins and parse the output for whatever purpose.

Full disclosure: I work for CA who created Brightside but we do use this CLI internally for the exact purpose of integrating our mainframe application testing with Jenkins.

Sujay Solomon
  • 11
  • 2
  • Have you used it? – chicks Jun 12 '18 at 18:11
  • I fail to see how this may answer the question even remotely, please [edit] your answer to explain how it can answer the question or I'll delete it in 24h – Tensibai Jun 12 '18 at 21:36
  • Sure, I'll add more detail to my post. – Sujay Solomon Jun 13 '18 at 14:26
  • One more thing, please disclose your affiliation with the company you're promoting. See the self-promotion paragraph in our behavior page. This sounds like an honest attempt to answer, I'm feel it miss the point of using a plugin integrated in jenkins for pipeline use, but at least it doesn't look like 'product promotion only' anymore. – Tensibai Jun 13 '18 at 15:50
  • TL;DR: answering with a product you own/create is ok when you disclose your link with the product and explain how it can solve the problem. – Tensibai Jun 13 '18 at 15:53
  • Ah, understood - thanks! It's a bit of odd situation here because we built this for our dev team's use to satisfy this exact case of needing to test/automate mainframe things from Jenkins/other testing frameworks like MochaJS. I will update the post to disclose that I do work for CA. – Sujay Solomon Jun 13 '18 at 15:56
  • As far as plugin vs cli approach for Jenkins.

    What we do is install brightside (and other CLIs/dependencies) into a docker image and every time a CI run happens in jenkins, we run this docker instance and all scripts that run from jenkins pipeline within this container now have access to all the CLIs like Brightside.

    For us this was easier than trying to work plugins as we like the flexibiity/freedom of being able to script in shell/python for mainframe actions.

     – Sujay Solomon Jun 13 '18 at 16:03
1

@Pierre.Vriens, wrt your ask about “investigate the feasibility of managing data stored in personal files (PDSes, i.e. partitioned data sets) on the mainframe.”, you can use zOS Management Facility (zOSMF) which comes with the z/OS operating system to access APIs for that. In Sujay's reply, he suggests using Brightside and that tool taps into the zOSMF APIs to simplify that task. So then if you want to tie Jenkins to the Mainframe to manage Partitioned Data Sets (PDS) and Data Sets (PS), you can use these zOSMF APIs or you could use Brightside.

In the spirit of disclosure, I work as a UX Designer for CA Technologies who builds Brightside.

John laPlante
  • 11
  • 2