Shop Mobile More Submit  Join Login
Vector Mask Painting by MissNysha Vector Mask Painting by MissNysha
This is a guide for those who want some more in depth clarification about my fav digital coloring technique: vector mask painting, which combines vectoring and tablet painting in a single layer. This time, you don't have to hunt through my other tutorials to try and understand the concept! :D

Intermediate level, as it assumes some photoshop vectoring knowledge. See my previous Ultimate vectoring guide for further details: [link]

Also check out my Hair Coloring Tutorial to see me using vector mask painting in full! [link]

Made in photoshop cs2, but the technique still applies to other versions. :)

Enjoy!

All parts made by me - No stealing or ripping any of its content.
Add a Comment:
 
:iconmelikebat:
MelikeBAt Featured By Owner Jun 25, 2014  Hobbyist Traditional Artist
This is awesome! Thank you!
Reply
:iconmissnysha:
MissNysha Featured By Owner Jun 26, 2014  Professional General Artist
:D Glad you like it!
Reply
:icongalenia:
Galenia Featured By Owner Jan 21, 2014  Hobbyist Interface Designer
Omg you are perfect! *-*
Reply
:iconchenillelily:
ChenilleLily Featured By Owner Jul 11, 2013
Hey there, sorry for bothering again. I have a question, I've just tried to group all the layers and add a group vector mask to it but when I was about to apply the gradient tool, it said "Could not use the gradient tool because the pixels in a group cannot be edited without merging the group"
Could you please help me with that?
Thank you.
Reply
:iconmissnysha:
MissNysha Featured By Owner Jul 13, 2013  Professional General Artist
Please PM me a note with a screenshot please. :)
Reply
:iconchenillelily:
ChenilleLily Featured By Owner May 29, 2013
Thank you!! This tutorial is really helpful, but still don't know how to make the colors so soft like yours. I have a question, when you paint a character, do you color the skin in a single layer (or vector mask) or make separated layers?
Thanks, again :)
Reply
:iconmissnysha:
MissNysha Featured By Owner May 29, 2013  Professional General Artist
:D
I make a new layer for each color so they can blend into each other softly (I mess around with the opacities and blending modes). I also use layer masks to erase around edges to "mold" colors into a nice soft shape too (with a large airbrush). I may merge these layers together later if the file size gets too big. All layers are grouped within a vector mask to keep me organized and their edges nice and clean. :)
Reply
:iconchenillelily:
ChenilleLily Featured By Owner May 30, 2013
I see. Oh the problem is, I don't know how to erase the edges, if you don't mind, [link] please take a look at the picture I'm working. May I know how to erase that part?

I'm sorry for my English.
Thank you very much.
Reply
:iconmissnysha:
MissNysha Featured By Owner Jun 1, 2013  Professional General Artist
Hmm let's see...
This could be caused by several things. Depends on how your layers are stacked too. If you have the same vector shape stacked in several layers, the outside edge develops a pixelly outline that can't be erased unless you group all the similar layers under a single mask (example of the bow at the bottom of the tutorial).
Also, if you used the paint bucket to fill in a pen tool path, and rasterized it, then the edges may become pixellated like this. I'd have to know what you did to make this, or see your layers to really give you a good answer.
Reply
:iconchenillelily:
ChenilleLily Featured By Owner Jun 2, 2013
I've figured it out, it's because I didn't erase the layer under the vector mask. Thanks for answering my question. =D
Oh I see, I'm new at vector mask so I still don't know how to use it. I want to make soft shading for the skin, clothes, and hair, but I don't know how to make them without making them pixellated. ^^; Should I use brush or gradient tool? And is it okay to use smudge tool? I'm afraid the edges may become pixellated if I use it.
Ah I'm so sorry I don't have the file anymore, I was only practicing and I didn't save it. :(
Reply
:iconmissnysha:
MissNysha Featured By Owner Jun 2, 2013  Professional General Artist
Good! Glad you figured it out. :)
I use both a large airbrush and the gradient tool. I use the gradient tool to apply lighting effects mostly though. I do not use the smudge tool on skin - only the airbrush, because the smudge tool can make some weird effects that do not make skin smooth. I use the smudge tool on hair though to paint the details.
Reply
:iconchenillelily:
ChenilleLily Featured By Owner Jun 3, 2013
So you only use airbrush and apply gradients for shading? If you don't mind, may I know how many layers do you usually use for skin color? And do you use tablet to paint?
Reply
:iconmissnysha:
MissNysha Featured By Owner Jun 3, 2013  Professional General Artist
That's right. Maybe about 10 layers or so, maybe more sometimes. And yes I use a Wacom tablet. :)
Reply
(1 Reply)
:iconkamame:
Kamame Featured By Owner Apr 30, 2013  Professional
Very interesting technique. It reminds me of what I do in Illustrator. In CS6 you get better painting brushes and vector masks really help a lot there, but I never thought about trying them in Photoshp..I'll definitely have to try that out some time! Thank you for taking the time to write this tutorial!
Reply
:iconmissnysha:
MissNysha Featured By Owner Apr 30, 2013  Professional General Artist
Glad to help! I do not have CS6 yet, but I am definitely intrigued by what the new Illustrator can do! Especially if the upgrade does not have a RAM limit, like how Photoshop CS5 currently runs (one of my favorite new features).
Reply
:iconkamame:
Kamame Featured By Owner May 15, 2013  Professional
I definitely have to check that out! I switched directly from CS4 to CS6, so I don't know much about the detailed changes. But I'm sure you'll enjoy the new features of CS6.. it's a blast! ^^
Reply
:iconvolcanic-penguin:
Volcanic-Penguin Featured By Owner Dec 25, 2012  Hobbyist Digital Artist
This is quite cool.

I make a lot of vector graphics in Flash, and would like to try out painting within those vector shapes, but I suppose then I'll have to settle for the magic wand tool, I hope that works fairly well without getting weird looking edges.
Reply
:iconmissnysha:
MissNysha Featured By Owner Dec 26, 2012  Professional General Artist
Actually I do not recommend the magic wand tool, because it selects a certain color base, with a lot of random fuzziness that results in pixellated edges. (unless you're only selecting 1 out of 2 colored areas) Using the pen tool to create vector masks is the solution to make the cleanest edges possible. It shouldn't be too much different from vectoring shapes in flash.

To rate the tools available, my selection/edges hierarchy goes like this:
Group vector mask > painted group layer mask > lock transparent pixels (when working on a single layer only) > polygon lasso tool > magic wand.

So you can see I only use the wand tool as a last resort. Although it may be the fastest, the randomness you can't control on its edges really brings the overall image quality down, which other viewers will easily notice. Plus once you make a mask, you only need to create it once, as opposed to having to reselect the area you want over and over, so in the long run it saves time too.

Is there a reason why you must settle on the wand tool?
Reply
:iconvolcanic-penguin:
Volcanic-Penguin Featured By Owner Dec 26, 2012  Hobbyist Digital Artist
Thanks for the response!

I've found that if you use the magic wand tool to select an area, you can then click on "Refine Edge" and then "Decontaminate Colors" and it will create a layer mask for your selection so that you won't have to re-select anything later. But you're right in that it's a bit messy.

Well the way vectors work in Flash is quite different from Photoshop or Illustrator and I find it more intuitive and flexible. It took a long time but I managed to figure out how to get the vectors from Flash to Photoshop. I had to export the Flash graphic as an fxg file, open it in Illustrator and save as an ai file, then open that in Fireworks and save as a psd file that when opened in Photoshop has every shape as a color fill and a vector mask.

The only issue with this is that if you have two adjacent vector shapes that share the same border edge, which you often have in Flash, you end up with a thin transparent line in between the two shapes in Photoshop, as if they're not quite touching. But I suppose I can just make sure shapes are always overlapping each other.
Reply
:iconsupersleuth10:
SuperSleuth10 Featured By Owner Mar 10, 2012  Hobbyist Digital Artist
very nice =)
i have one question though.
Why do you have a layer mask on all of your layers or more specifically what do you use it for?
Reply
:iconmissnysha:
MissNysha Featured By Owner Mar 10, 2012  Professional General Artist
I use layer masks separately from vector masks for non-destructive erasing. Basically I can soften and fade parts without permanently erasing them - and if I make any mistakes I can just paint back with white to bring parts back. :)
Reply
:iconsupersleuth10:
SuperSleuth10 Featured By Owner Mar 10, 2012  Hobbyist Digital Artist
interesting, I may just try that.
Thanks!
Reply
:iconminx188:
Minx188 Featured By Owner Feb 19, 2012
Hi What my question is if i did some work @ photoshop and drag my work into illustrato and vise versa is this the efficient way in terms of getting good quarlity and making working easier? Or do i need to save in some sort of format like like jpeg. or something to make the process smooth and correct- to one software into the other.

What's the difference betwee eps and ai. ?
Sorry about the lame question but im a complete beginner in the technical size of things.
Reply
:iconminx188:
Minx188 Featured By Owner Feb 26, 2012
Thank you for the tips :)
Reply
:iconmissnysha:
MissNysha Featured By Owner Feb 19, 2012  Professional General Artist
Well let's see. I typically only use photoshop most of the time, since all the tools are already there pretty much. (And illustrator tends to be buggy and freak out/crash sometimes) When vectoring in photoshop, you can try export>paths to illustrator, but I haven't tried that before. I have worked in illustrator and exported the file as a psd, which keeps all your layers (but everything still becomes rasterized in those layers). An .ai file is illustrator's basic save file, like how it's a psd for photoshop documents. Saving it as an .eps in illustrator keeps some vector qualities, because when you try to open it in photoshop it will ask you what size you want it at. However, once you choose a size, photoshop rasterizes the image at that size. Eps is used for stuff like print making and other related graphics. (However, if you use some fancy gradients and stuff, I have noticed some visible color/quality changes after exporting it as a new file like an eps. So just experiment and see if the quality drops) I would say avoid jpegs, and try pngs or even pdfs if you want a higher quality flattened image.

So unless you really need to, in my opinion it's better/easier/more efficient just to choose one program and stick with it for a project you're working on, instead of switching back and forth all the time. But if you want to switch around, it's better to start in illustrator to draw your lineart and basic shapes, then export it as a psd to add painting and special effects to it in photoshop.

I hope that helped!
Reply
:iconmairelyn:
Mairelyn Featured By Owner Feb 9, 2012  Hobbyist General Artist
Wow This looks like an Incredible technique!

Thanks for making this tutorial!!
Reply
:iconmissnysha:
MissNysha Featured By Owner Feb 9, 2012  Professional General Artist
Glad you found it useful! :D
Reply
:iconlaughingvulcan:
laughingvulcan Featured By Owner Jan 27, 2012  Hobbyist General Artist
Faved for future studying!

And THANKS for putting this up! :squee: :)

Be well.

Greg
Reply
:iconmissnysha:
MissNysha Featured By Owner Jan 27, 2012  Professional General Artist
Glad you like it! :D And best to you as well.
Reply
:icondraaky:
Draaky Featured By Owner Oct 24, 2011  Hobbyist Digital Artist
Requesting Video tutorial :3
Reply
:iconmissnysha:
MissNysha Featured By Owner Oct 25, 2011  Professional General Artist
If only I knew how! XD Not sure if CamStudio is good enough.
Reply
:iconclover31:
Clover31 Featured By Owner Aug 6, 2011
thank you soooo much for this tutorial!!! it has helped me so much :iconproudplz:
Reply
:iconmissnysha:
MissNysha Featured By Owner Aug 8, 2011  Professional General Artist
:D Glad to help!
Reply
:iconclover31:
Clover31 Featured By Owner Aug 8, 2011
i just have 1 question though, how do you get multiple vector selections on 1 layer?
Reply
:iconmissnysha:
MissNysha Featured By Owner Aug 8, 2011  Professional General Artist
To select multiple shapes in the same layer, use the black or white arrow and press "shift" as you click them. :3 I usually do this while using the pen tool and pressing "ctrl"+"shift" to get the white arrow.
Reply
:iconclover31:
Clover31 Featured By Owner Aug 11, 2011
thanks! :D
Reply
:iconskychainer:
SkyChainer Featured By Owner Jul 22, 2011  Student General Artist
thx 4 the tutorial ^^ I'm a beginner but I think im going 2 try vectors now, although im a bit confused ill eventually get it ^^ one question though the blending in the picture looks very smooth what did you use? and you do your lineart first then vector the stuff on top of it right? (hiding the lineart?)
Reply
:iconmissnysha:
MissNysha Featured By Owner Jul 22, 2011  Professional General Artist
I used a large airbrush inside the vector masks, so I didn't need to use any lineart either. :D Download this to see what I mean: [link]
Reply
:iconskychainer:
SkyChainer Featured By Owner Jul 22, 2011  Student General Artist
thx 4 replying and sharing ^^
Reply
:iconholyeye:
HolyEyE Featured By Owner Jul 12, 2011
OMG dude you are awesome! I wanted to create my own manga and i didnt know how they make its quality so high! now i can do it!
thank you so much :))))
Reply
:iconmissnysha:
MissNysha Featured By Owner Jul 12, 2011  Professional General Artist
Glad to help! :D
Reply
:iconmysticdarkangel:
MysticDarkAngel Featured By Owner Jul 9, 2011
I had heard of vector mask painting, but I was clueless as to how to even begin. :dizzy:
Thank you very much for this tutorial, as well as explaining it in a relateable fashion. ;D
Reply
:iconmissnysha:
MissNysha Featured By Owner Jul 10, 2011  Professional General Artist
XD Hehehe glad you liked the tut!
Reply
:iconsaru-chikin:
saru-chikin Featured By Owner Mar 31, 2011  Hobbyist General Artist
Nysha +__+ Awesome tutorial, which shows me, that I am an BAKA!
I always use this method for vectoring ONLY, never thought about painting on the layers >_>
Never thought...that it was even possible 3
Thank you so much, I will try that next time!
Always love your work! ALWAAAYS ><

(chrisp MT/pulver AP)
Reply
:iconmissnysha:
MissNysha Featured By Owner Mar 31, 2011  Professional General Artist
XD Hehehe, Photoshop has lots of neat tricks doesn't it? Just takes some experimenting to try out different combinations. :3
Good luck! Would like to see what you make next time! :D
Reply
:iconsaru-chikin:
saru-chikin Featured By Owner Apr 1, 2011  Hobbyist General Artist
Chikin:
I will! >3 And then I will show ya, promise!
Kekeke :iconhurrplz:
Reply
:icontoushi-kun:
Toushi-Kun Featured By Owner Mar 14, 2011  Student Digital Artist
I really am a fan of your works. :XD:

Kudos for the great job!

:+fav:
Reply
:iconmissnysha:
MissNysha Featured By Owner Mar 14, 2011  Professional General Artist
Thank you! :D
Reply
:icontoushi-kun:
Toushi-Kun Featured By Owner Mar 17, 2011  Student Digital Artist
AS always, you're very welcome! :D
Reply
:iconwarwickscomicart:
WarwicksComicArt Featured By Owner Feb 23, 2011  Hobbyist Digital Artist
This is great very well done
Reply
Add a Comment:
 
×
Download JPG 1000 × 6229




Details

Submitted on
February 17, 2011
Image Size
1.3 MB
Resolution
1000×6229
Link
Thumb
Embed

Stats

Views
23,919 (7 today)
Favourites
906 (who?)
Comments
80
Downloads
962
×